JP2001217850A - Method for realizing user interface - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a user interface(UI) realizing method capable of providing a common user interface regardless of a programming language of an operating subject. SOLUTION: This UI realizing method includes a step in which a UI description document 1102 where UI elements that should constitute the user interface between the operating subject 1101 of an application program, etc., and a user are described by using logical UI elements obtained by abstracting the functions of the UI elements in interaction with the user is read and analyzed, a step 1201 in which the document 1102 is referred and a UI between the operating subject and the user is constructed by using concrete UI elements corresponding to the described logical UI elements, and a step in which a 1st message is generated in response to a user operation with respect to the concrete UI elements included in the user interface and transmitted (message communication) to the operating subject.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and system for implementing a user interface when a user operates an application program or controls one or more devices in a computer system. In particular, the present invention relates to a method and system for realizing a user interface capable of separating a user interface from an application program or a processing part (operator) that performs actual processing such as a device.

[0002]

2. Description of the Related Art Due to recent advances in network technology,
It is a matter of course that data stored in another computer terminal connected to the network is referred to and operated from the other terminal. Recently, IEEE
By using a bus connection interface such as 1394, it is possible to connect not only computers but also household appliances to a network. As a result, a framework is being constructed in which a user can freely operate a plurality of devices using an arbitrary operation device such as a remote control.

In such a network environment,
In any device, it is necessary that a control program or an application program of the device can be operated using any device capable of providing a user interface (hereinafter, referred to as “UI”). is there. Such a problem is achieved in the field of computer systems by separating the program main body from the UI.

[0004] As a typical one of the conventionally proposed methods, there is a method in which all programs are unified using the same programming paradigm. For example, a client / server model such as an X Window system in a UNIX system is a framework in which an execution process of a program body and a window manager process for providing a UI can be managed separately. With this framework, the program operates properly even if the device on which the program itself is running and the device providing the UI are different from each other.

When using the programming language Java, as long as all devices have a Java execution environment,
Any device can download and execute the program of another device or a part thereof. The Java language further employs the MVC paradigm for GUI (Graphical User Interface) programming. The MVC paradigm is a paradigm first adopted in the programming language Smalltalk-80. The MVC paradigm includes a program (M) composed of data and its processing procedure, a view (V) for presenting data to a user, and It is configured using three controllers (C) that accept user input for operation. According to the MVC paradigm, a narrowly-defined UI composed of a view and a controller directly contacted by a user and a model unique to a program can be separately expressed.

On the other hand, JP-A-5-27959 discloses that
It proposes a method that can absorb the difference even when a plurality of programs use different GUI toolkits. In the technology disclosed in this publication, a plurality of GUIs
The concept of common parts is applied to GUI parts defined in the toolkit, and a correspondence table between individual GUI parts and common parts is created. According to this correspondence table, even if a device that should provide a UI can provide a UI only with a specific GUI toolkit, if a GUI part (toolkit) is converted using the correspondence table, it can be used in an arbitrary program. User interface can be provided to the user.

Further, HTML (Hyper Text Markup Languag)
There is also a method of separately describing a program body and a UI by using e) and CGI (Common Gatewey Interface). This method is a method generally performed in WWW (World-Wide Web), and stores a program body called a CGI program and a UI described in HTML in a server device. The user can obtain the UI by downloading the HTML file using any Internet browser. Then, the CGI program can be controlled through the UI.

The greatest advantage of this method is that the UI can be provided by any Internet browser by using a general-purpose and logical GUI component defined in HTML.

[0009]

According to the method of unifying all programs using the same programming paradigm, in a world closed by the programming paradigm, it is possible to separate the program body and the UI. It is. However, there is no guarantee that future home networks will be developed with only one programming paradigm. Also, the history of the development of computer systems shows that it is very unlikely that a certain programming paradigm will be used forever.

The technique described in Japanese Patent Laid-Open No. 5-27959 is said to be applicable to different programming paradigms. However, if the programming paradigm is different, the communication interface between the program body and the UI is also different, so that it is expected that this technique will be quite difficult to realize. Thus, where the techniques described in this publication can be readily applied, there are certain systems where a particular programming paradigm, especially
Probably only when closed to the X Window System.

In the technology using HTML and CGI, it can be said that a general-purpose and logical GUI component is defined to some extent. However, the essence of HTML is to mark up the document to lay out the document, and the GUI components provided there are only additional. Therefore, the functions and roles of individual GUI components are not systematically defined. Also, due to the nature of HTML that allows documents to be viewed with the same layout by any browser, the GUI components used in this method are only static ones that cannot be arbitrarily extended by the writer.

Accordingly, an object of the present invention is to provide a user interface realizing method capable of providing a common user interface irrespective of a programming language in which an operation subject such as an application program or a control program is realized. It is to provide.

[0013]

According to a first aspect of the present invention, there is provided a user interface realizing method for an apparatus for providing a user interface to a user, the method comprising: Reading and analyzing a user interface description document that describes a user interface element that should constitute a user interface between the user interface element and the logical user interface element that abstracts the function of the user interface element in the interaction with the user;
Referring to the read user interface description document and configuring a user interface between the action subject and the user using the concrete user interface element corresponding to the described logical user interface element; Performing a process for generating a first message in a predetermined first format and transmitting the first message to the operating entity in response to a user operation on the user interface element.

By describing a logical user interface element in a user interface description document and reading it, a user interface can be configured in accordance with the description and a user's instruction can be transmitted to an operation subject. Since the action subject and the user interface description document are separate, and the user interface element is described using a logical user interface element that abstracts the function of the user interface element in the interaction with the user, the user interface It is possible to provide a versatile user interface that can be used even if the device that implements is different, and can be used even if the programming language used when the operation subject is a program is different.

According to a second aspect of the present invention, in addition to the configuration of the first aspect, the user interface realizing method further receives a second message in a second format predetermined from an operation entity. And modifying the user interface in response to the received second message.

[0016] Since the user interface is changed by the second message from the subject, a complete interaction between the user and the subject can be realized.

According to a third aspect of the present invention, in addition to the configuration of the second aspect, the user interface description document defines a concrete user interface element corresponding to the logical user interface element. Changing the user interface may include updating, in response to the received second message, a logical user interface element that has received the second message; and updating the logical user interface element with the updated logical user interface element. And updating the concrete user interface element corresponding to.

A concrete user interface element corresponding to a concrete user interface element corresponding to a logical user interface element required for an operation of an operation subject is defined,
When the logical user interface element is updated by the second message from the operator, the concrete user interface element is updated accordingly. Therefore, the interaction between the user and the action subject can be easily performed through the concrete user interface element.

According to a fourth aspect of the present invention, in addition to the configuration of the first or second aspect, the user interface description document includes information on one or more logical user interface elements. It is a written document.

A user interface can be configured using a plurality of logical user interface elements.

According to a fifth aspect of the present invention, in addition to the configuration of the first aspect of the present invention, the logical user interface element further comprises a function and a corresponding function. Is defined to form a hierarchical classification tree that is abstracted and hierarchized at a plurality of levels of abstraction in accordance with the degree of abstraction.

Since the logical user interface elements are defined so as to form a classification hierarchy tree according to the degree of abstraction, the logical user interface elements located close to each other on the classification hierarchy tree realize similar functions. It is thought that it can be done. for that reason,
For example, when a certain logical user interface element cannot be realized in a certain user interface device, by using a logical user interface element that can be realized in a position close to the logical user interface element on the classification hierarchical tree, it is possible to substantially perform the processing. A user interface having a similar function can be provided to the user.

According to a sixth aspect of the present invention, in addition to the configuration of the fifth aspect, the description of the logical user interface element of the user interface description document includes another description in the user interface description document. Having a description defining an identifier for distinguishing the logical user interface element from the self and a description defining a value indicating a state of the logical user interface element, and providing a corresponding identifier to the concrete user interface element And providing a corresponding value to the concrete user interface element.

By specifying a logical user interface element with an identifier and giving a value, the state of the logical user interface element can be changed.

Preferably, the description of the logical user interface element further includes a description defining one or more value candidates that can be used as the value of the logical user interface element, and providing the concrete user interface element with a corresponding value candidate. In addition.

According to a seventh aspect of the present invention, in addition to the configuration of the sixth aspect of the present invention, the description of the logical user interface element is further assigned to the value and the value candidate. And assigning a corresponding name to the value and the value candidate given to the concrete user interface element.

By assigning corresponding names to the values and value candidates of the logical user interface element, the names can directly represent the contents of the values or the value candidates, and different names can be given names having a predetermined relationship. By doing so, it is possible to give a certain relationship between those values.

According to an eighth aspect of the present invention, in addition to the configuration of the first aspect of the present invention, a first attribute is defined for a certain logical user interface element. , The first attribute is inherited by lower logical user interface elements that derive from one logical user interface element in the classification hierarchy tree.

Since the first attribute is inherited by lower logical user interface elements, the similarity of each logical user interface element on the classification hierarchy tree is maintained, and the description of the user interface description document is maintained. Also becomes easier.

Preferably, the first attribute of a certain logical user interface element is fixed to a certain value, and the first lower logical user interface element derived from the certain logical user interface element in the classification hierarchy tree includes: Fixing the attribute to a certain value.

According to a ninth aspect of the present invention, in addition to the configuration of the first aspect of the present invention, the logical user interface element includes a function of the logical user interface element. Has an independent second attribute.

Since the second attribute independent of the function of the logical user interface element can be given to each logical user interface element, the shape of the concrete user interface element corresponding to each logical user interface element,
Properties that are not directly related to the function, such as color, can be controlled by the description in the user interface description document.

Preferably, the information on the logical user interface element described in the user interface description document further includes a name representing the type of the logical user interface element, and a value required according to the type of the logical user interface element. Information about the candidate and information about the first attribute.

More preferably, the information on the logical user interface element described in the user interface description document further includes information on the second attribute when required by the type of the logical user interface element.

According to a tenth aspect of the present invention, in addition to the configuration of the first aspect, at least one of the logical user interface elements includes one. This is a container-type logical user interface element that can include the other logical user interface elements described above.

By introducing a container type logical user interface element, for example, the logical user interface elements can be grouped according to a certain relationship, and the operability of the user interface is improved.

[0037] Preferably, the step of changing the user interface comprises: a logical user interface element updating step of updating a logical user interface element described in the user interface description document based on the second message; A concrete user interface element updating step of updating a concrete user interface element associated with the logical user interface element based on the logical user interface element updated by the updating step, wherein the second message A value change message instructing to change the value of the interface element; a value candidate change message instructing to change the value candidate of the logical user interface element; The first to change the attribute value of an attribute
And a second attribute change message for changing the attribute value of the second attribute of the logical user interface element.

[0038] Preferably, the logical user interface element updating step includes, in response to the second message being a value change message, changing the value of the logical user interface element having the identifier included in the second message into the second message. And updating with a value included in the message.

Preferably, when the second message is a value candidate change message, the second message includes a value,
And information indicating the type of change. The logical user interface element updating step further comprises: in response to the second message being a value candidate change message, the logical user interface element having an identifier included in the second message. Updating the logical user interface element by performing operations on the element indicated by the value specified by the second message and the type of change included in the second message.

Preferably, the type of change included in the value candidate change message is either addition or deletion of a value candidate or change of the value of the value candidate, and the step of updating the logical user interface element includes the step of: In response to the message being a candidate value change message and the type of change included being the addition of the candidate value, a second message is generated for the logical user interface element having the identifier included in the second message. Adding the value contained in the second message to the second message in response to the second message being a value candidate change message and the type of change being included being deletion of the value candidate. Deleting a value candidate that matches the value included in the second message for the logical user interface element having the identifier And a flop.

Preferably, the step of updating the logical user interface element includes the step of updating the logical user interface element having the identifier included in the second message when the type of change included in the second message is a change in the value of the value candidate. The method further includes the step of replacing the value of the value candidate that matches the name of the value included in the second message with the value included in the second message.

Preferably, the first attribute change message includes an identifier of the logical user interface element, a name and an attribute value of the first attribute, and the logical user interface element updating step includes the step of: First attribute matching the name of the first attribute included in the second message for the logical user interface element having the identifier included in the second message in response to the first attribute change message To the attribute value included in the second message.

Preferably, the second attribute change message includes an identifier of the logical user interface element, a name and an attribute value of the second attribute, and the step of updating the logical user interface element includes the step of: In response to the second attribute change message, the second
For the logical user interface element having the identifier included in the second message, the attribute value of the second attribute that matches the name of the second attribute included in the second message is set to the attribute value included in the second message. Is included.

Preferably, the step of generating the first message and transmitting the message to the operation subject is associated with the concrete user interface element in response to a user operating the concrete user interface element. Updating the value possessed by the logical user interface element, and generating a first message using the identifier and the value possessed by the logical user interface element.

Preferably, the logical user interface element corresponding to a certain concrete user interface element has a plurality of value candidates, and the step of updating is performed in response to a user operating a certain concrete user interface element. Updating the value of a logical user interface element with one or more of the value candidates.

Preferably, the method further includes the step of updating the value of the concrete user interface element associated with the updated logical user interface element with the updated value of the logical user interface element.

Preferably, one logical user interface element is a container-type logical user interface element that can include one or more other logical user interface elements, and the step of generating the first message comprises the step of: Receiving a first message generated by another logical user interface element when a user operates a concrete user interface element corresponding to another logical user interface element included in the interface element, and the received first message Generating an identifier list in which the identifiers included in the first message and the own identifier are arranged in a predetermined order, and the first message further updated using the generated identifier list and the value included in the first message Generating.

According to an eleventh aspect of the present invention, in addition to the configuration of the fifth aspect, the user interface description document includes a definition relating to a description format of the user interface description document and a logical user interface. It is based on user interface description document definition information that describes information about elements and definitions about their description styles, and information about classification hierarchy trees.

Since the user interface description document is described based on the definition of the description format of the user interface description document, the description content in the user interface description document can be changed by changing the description format. As a result, an optimal user interface can be realized by defining a description format according to the processing to be performed by the operation subject and describing the user interface description document in accordance with the format.

Preferably, the step of reading and analyzing comprises:
Reading the user interface description document and the user interface description document definition information; analyzing the read user interface description document based on the read user interface description document definition information; Verifying whether or not they have been performed.

Preferably, the classification hierarchy tree includes a tree structure of a plurality of logical user interface elements rooted at one abstract logical user interface element, and a logical user interface element corresponding to an arbitrary node in the tree structure The features defined for
It branches from the node and is inherited by logical user interface elements corresponding to all nodes lower than the node.

Preferably, the features defined for the logical user interface element include: a concrete user interface element update function for initializing or updating the concrete user interface element associated with the logical user interface element; A value update function for updating the value of the logical user interface element based on an operation performed by the user on the concrete user interface element associated with the logical user interface element, an identifier of the logical user interface element, And a message propagation function for generating a third message including a value and propagating to a predetermined destination logical user interface element when a predetermined condition is satisfied. 4th received
Any combination of a logical user interface element update function that updates itself in response to the message of the above, and a value reference function that updates the value of the logical user interface element by referring to the value of another logical user interface element including.

[0053] Preferably, the step of changing the user interface includes: a logical user interface element updating step of updating a logical user interface element described in the user interface description document based on the second message; Updating a concrete user interface element associated with the logical user interface element based on the logical user interface element updated by the updating step. Preferably, the second message has a plurality of types.

According to a twelfth aspect of the present invention, in addition to the configuration of the second aspect, one logical user interface element may include one or more other logical user interface elements. Container-type logical user interface element, wherein the second message includes an identifier of the logical user interface element for identifying the logical user interface element in the container-type logical user interface element, wherein the identifier is the container-type logical user interface. The method further comprises the step of receiving a second message, comprising: a list of identifiers arranged in the container type logical user interface element according to a hierarchical structure in which the identifiers of the logical user interface elements in the element are arranged. Received Determining whether the own identifier is included in the identifier list included in the second message; and determining that the own identifier is not included in the identifier list included in the received second message. Ending the process in response to the request; and deleting the own identifier from the identifier list in response to the determination that the identifier list included in the received second message includes the own identifier. Determining whether the identifier list after deleting its own identifier is empty, and changing the user interface,
Updating the self in accordance with the content of the received second message in response to the identifier list after removing its own identifier being determined to be empty,
The method further includes, in response to determining that the list of identifiers after deleting the identifier of the user is not empty, including a second message including the list of identifiers after deleting the identifier of the other Sending to the logical user interface element.

By updating the identifier list of the second message in this way, it becomes easy to search for the logical user interface element to which the second message is to be propagated.

Preferably, a predetermined message propagation destination and a predetermined message propagation source are operation entities.

Preferably, the step of constructing the user interface is read and analyzed based on a prepared user interface element correspondence table which describes correspondence between logical user interface elements and concrete user interface elements. Associating a logical user interface element described in a user interface description document with a concrete user interface element prepared in an apparatus for providing a user interface, and a logical user interface described in the user interface description document Generating and initializing a logical user interface element using information about the element; and, associated with the logical user interface element based on the initialized logical user interface element. And comprising initializing a concrete user interface element, and providing a user interface to a user by using an initialized concrete user interface element.

Preferably, the associating step is a step of judging whether or not a correspondence relation regarding the logical user interface element described in the user interface description document exists in the user interface element correspondence table. According to the result, if the correspondence exists, the concrete user interface element defined by the user interface element correspondence table is used. If the correspondence does not exist, the concrete user interface element has the predetermined relation with the logical user interface element in the classification hierarchy tree. A step of determining a concrete user interface element corresponding to a certain logical user interface element as a concrete user interface element corresponding to the logical user interface element described in the user interface description document. And a flop.

Preferably, the logical user interface element having a predetermined relationship is located above the logical user interface element described in the user interface description document in the classification hierarchy tree and is the closest logical user interface element or the closest logical user interface element. Another user interface derived from the logical user interface element.

[0060] Preferably, the logical user interface element has one or more value candidates, and the step of performing processing for transmission to the operation subject is responsive to a user operation on a concrete user interface element included in the user interface. And selecting one of the one or more value candidates of the logical user interface element as the value of the logical user interface element.

According to a thirteenth aspect of the present invention, in addition to the configuration of the first aspect, the user interface realizing method according to the present invention further comprises:
The user interface description document defines whether the user has the message transmission right, which is the right to transmit the first message to a predetermined destination, or is delegated to another, and performs a process for transmitting the first message to the operation subject. Generating a first message in response to a user operation on a concrete user interface element corresponding to the logical user interface element included in the user interface; and determining whether the logical user interface element has a message transmission right. Determining whether to send the first message to a predetermined destination according to whether it is determined that the logical user interface element has the right to send a message.

Even when the user operates a concrete user interface element corresponding to a certain logical user interface element and the value of the logical user interface element changes as a result, the change is mainly performed by the operation each time. May not need to be sent. For example, when the value of another logical user interface element changes, it may be transmitted at the same time. In such a case, by delegating the above-mentioned message transmission right to another logical user interface element in advance, the other logical user interface elements can change their own values when transmitting a message. Information can also be sent. As a result, traffic for message communication can be reduced.

Preferably, for each logical user interface element, the user interface description document has the right to send the first message to a predetermined destination, a message sending right, or is delegated to another. And, if delegated, the destination of the delegation is specified, and information for specifying the logical user interface element of the delegation source of the message transmission right is transmitted to the logical user interface element of the delegation destination of the message transmission right. The method further includes the step of, in the logical user interface element of the delegation destination, storing information specifying the logical user interface element of the delegation source that has delegated the message transmission right to the logical user interface element of the delegation destination.

Preferably, for each logical user interface element, in the user interface description document, a message transmission right that is a right to transmit the first message to a predetermined destination is transmitted from another delegation source logical user interface element. Whether or not the information has been delegated, and information for specifying the delegation source logical user interface element is defined, and the step of performing processing for transmitting to the operation subject is performed in the logical user interface element included in the user interface. Generating a first message in response to a user operation on a corresponding concrete user interface element; determining whether the logical user interface element has been delegated message transmission rights from others; The interface element is a message In response to the determination that has been delegated Shin rights, including the steps of delegating original message transmission right is added to the first message reads the messages it contains.

Preferably, the user interface description document can define a predetermined feature for each logical user interface element, the feature being that the logical user interface element has a logical user interface element update function, The logical user interface element update function has a function of updating the logical user interface element according to the content of the second message in response to the second message propagated and received from a predetermined message propagation source. The method further comprises the step of determining a type of the second message, and, based on the determined type of the second message, a second one of the properties associated with the logical user interface element. Update according to the content of the message And a step.

Preferably, the predetermined message propagation source is an operation subject. Preferably, the logical user interface element has, as properties associated with it, a value of the logical user interface element and an identifier for specifying the logical user interface element, and the second message includes an identifier of the logical user interface element. And a value, wherein the method further comprises the step of receiving a second message at the logical user interface element, wherein the identifier included in the received second message is the logical user who received the second message. Determining whether the identifier of the logical user interface element that received the second message is equal to the identifier of the interface element; responsive to determining that the identifier of the logical user interface element that received the second message is equal to the identifier included in the second message; The logical unit that received the second message The values included in the The interface element, and a step of changing the values contained in the second message.

Preferably, the logical user interface element has, as properties associated with it, the value of the logical user interface element, one or more value candidates that the logical user interface element can have, and the logical user interface element. Each value candidate has a unique name, and the second message includes a logical user interface element identifier and one or more value candidate names. The user interface realization method further comprises a second user interface element in the logical user interface element.
Receiving the second message; determining whether an identifier included in the received second message is equal to the identifier of the logical user interface element that received the second message; In response to the determination that the received identifier of the logical user interface element is equal to the identifier included in the second message, the value of the logical user interface element that received the second message is stored in the second message. To a value of a value candidate having a name that matches the name included in.

Preferably, the logical user interface element has, as properties associated with it, the value of the logical user interface element, one or more value candidates that the logical user interface element can have, and the logical user interface element. Each of the value candidates has a unique name. The second message includes an identifier of the logical user interface element, a name of the value candidate, and a second message. Type information indicating the type of the second message, the method further comprising the step of receiving a second message at the logical user interface element, wherein the identifier included in the received second message indicates that the second message Whether it is equal to the identifier of the received logical user interface element Determining the logical user interface element receiving the second message in response to determining that the identifier of the logical user interface element receiving the second message is equal to the identifier included in the second message. Performing a process specified by the type information and the name included in the second message on one or more value candidates of the interface element.

Preferably, the type information can include information indicating addition of a value candidate to one or a plurality of value candidates, and the second message includes a value, and the type information included in the second message is included. The step of performing the process specified by the information includes, in the value candidate of the logical user interface element that has received the second message, having a name included in the second message and a value included in the second message Adding a value candidate.

Preferably, the type information may include information indicating deletion of a value candidate from one or a plurality of value candidates, and the step of performing a process specified by the type information included in the second message is performed. , Removing the value candidate having the name included in the second message from the one or more value candidates of the logical user interface element receiving the second message.

Preferably, the type information can include information indicating a change of one of the one or more value candidates, and the second message includes a value and is included in the second message. Performing the process specified by the type information, the value of the value candidate having the name included in the second message among the one or more value candidates of the logical user interface element receiving the second message To the value included in the second message.

Preferably, in the user interface description document, attributes relating to a certain logical user interface element can be defined, and each attribute is given a name, and the logical user interface element specifies the logical user interface element Has an identifier, and
As an attendant property, it has an attribute value of an attribute of the logical user interface element, and the second message is
The method further comprises receiving a second message at the logical user interface element, comprising the identifier of the logical user interface element, the name of the attribute, the attribute value of the attribute, and the received second message. Determining whether the identifier included in the second message is equal to the identifier of the logical user interface element that has received the second message; and determining whether the identifier of the logical user interface element that has received the second message is included in the second message. In response to the determination that the attribute is equal to the included identifier, the attribute of the attribute having the name matching the name of the attribute included in the second message among the attributes of the logical user interface element that received the second message. Changing the value to a value included in the second message.

Preferably, the logical user interface elements are defined so as to form a classification hierarchical tree hierarchized according to the function of each and the degree of abstraction of the function, and the attribute is defined in the classification hierarchical tree. It is derived from a certain logical user interface element and is inherited by lower logical user interface elements.

According to a fourteenth aspect of the present invention, in addition to the configuration of the first aspect, an attribute relating to a certain logical user interface element can be defined in a user interface description document. Has a value providing function of providing a value possessed by the logical user interface element to another logical user interface element of a value reference source, and provides a value from another logical user interface element of a value reference destination A value reference processing function of updating a self-value by applying a predetermined value reference model to the value to be performed.

With the value reference function, a user operation that is not directly related to the function realized by the operation subject can be processed only inside the device that provides the user interface.
Since there is no need to send a message to the actor, traffic for message communication can be reduced. Further, the change of the value in a plurality of logical user interface elements can be concentrated on the value of one logical user interface element.

Preferably, the attribute is an attribute independent of the function of the logical user interface element.

Preferably, the logical user interface element has, as properties associated with it, a value providing function of providing a value possessed by the logical user interface element to another logical user interface element of a value reference source, A value reference processing function of updating a self-value by applying a predetermined value reference model to a value provided from a logical user interface element of a value reference destination. Is also
Providing, at a predetermined timing, a value from the logical user interface element of the value reference source to the logical user interface element of the value reference destination; and responding to the provision of the value from the logical user interface element of the value reference destination. And updating the own value by applying a predetermined value reference model to the provided value in the logical user interface element of the value reference source.

Preferably, the step of providing the value includes the step of changing the value of the logical user interface element of the value reference destination from the logical user interface element of the value reference destination at the timing when the value of the logical user interface element of the value reference destination is updated. Providing a value.

Preferably, the step of updating its own value includes, in response to the provision of the value from the logical user interface element to which the value is referred to, the value provided in the logical user interface element of the value reference source. Substituting its own value.

Preferably, the step of updating the own value further includes the step of generating and propagating a message using the updated own value.

Preferably, the step of updating the own value further includes the step of updating the concrete user interface element corresponding to the own by using the updated own value.

Preferably, the step of updating its own value includes, in response to the provision of the value from the logical user interface element to which the value is referred to, in the logical user interface element of the value reference source, Replacing the value of the predetermined function with its own value.

Preferably, the logical user interface element includes a container type logical user interface element which may include another logical user interface element, and the step of performing a process for transmitting to the operation subject generates the first message. The logical user interface element is
Transmitting the first message to the other logical user interface element if the first message is included in another container-type logical user interface element; otherwise, transmitting the first message to the operation subject including.

Preferably, the method further comprises the steps of: receiving a second message in a predetermined second format from the operating entity; and changing the user interface in response to the received second message. Including.

Preferably, the step of receiving the second message includes the step of transmitting the second message to the container-type logical user interface element when the logical user interface element receiving the second message is a container-type logical user interface element. And determining otherwise that the second message is a message addressed to the logical user interface element that received the second message.

Preferably, the logical user interface element includes a container-type logical user interface element which may include other logical user interface elements, wherein the logical user interface element is transmitted from a logical user interface element included therein. Transmitting a message transmitted to a predetermined destination, and transmitting a message transmitted from a transmission source to a logical user interface element included in the container type logical user interface element. .

Preferably, the step of transmitting to a predetermined destination includes adding an identifier of the logical user interface element to an identifier of the logical user interface element included in the message transmitted from the logical user interface element included therein. Transmitting to a predetermined destination.

Preferably, the step of transmitting to the logical user interface element included therein includes deleting the own identifier from the identifier included in the message transmitted from the transmission source and storing the message inside the self. Sending to the included logical user interface element.

Preferably, for the logical user interface element, the user interface description document has the right to send the first message to a predetermined destination, or has been delegated to another, and If the message transmission right is delegated, information for specifying the logical user interface element to be delegated can be specified, and the step of configuring the user interface is performed by the logical user interface described in the user interface description document. Generating and initializing a logical user interface element using information about the element, and a message transmission right transfer source based on a message transmission right rule described in a user interface description document for the logical user interface element. logic Generating a message transmission right delegation relationship between the user interface element and the logical user interface element of the message delegation destination to which the message transmission right has been delegated. Generating a first message of a predetermined first format at a delegation destination in response to a user operation on a concrete user interface element included in the user interface;
Performing a process of adding the message generated by the logical user interface element of the transfer source of the message transmission right to the first message and transmitting the message to the operation entity based on the message transmission right transfer relationship.

Preferably, for the logical user interface element, another logical user interface element can be specified as a value reference destination of the value in the user interface description document, and the step of configuring the user interface is as follows. In addition, based on the provisions regarding the reference of the value described in the user interface description document regarding the logical user interface element, the reference logical user interface element that refers to the value of another logical user interface element and the value reference destination The method further includes the step of generating a value reference relationship with a certain logical user interface element, and the step of performing a process for transmitting the value reference relationship to the operation subject includes the step of: Reference the value at a predetermined timing Further comprising the corresponding steps of updating the value of the logical user interface elements values of values reference source logical user interface elements.

Preferably, for the logical user interface element, another logical user interface element can be specified as a value reference destination of the value in the user interface description document, and the step of configuring the user interface is as follows. Generating and initializing a logical user interface element using information on the logical user interface element described in the user interface description document, and referring to a value described in the user interface description document with respect to the logical user interface element Generating a value reference relationship between a reference logical user interface element that references a value of another logical user interface element and a logical user interface element to which the value is referred based on a rule, Performing a process for transmitting to the body, the logic interface elements, values referenced, in response to a user operation on concrete user interface elements included in the user interface,
Changing the value of the logical user interface element of the value reference destination, and responding to a user operation on the concrete user interface element included in the user interface, based on the value reference relationship, Updating the corresponding value of the logical user interface element of the value reference source with the value.

Preferably, a concrete user interface element update function for updating a concrete user interface element associated with a logical user interface element can be defined in a user interface description document, and a value reference destination can be defined. In the step of changing the value of the logical user interface element, the concrete user interface element corresponding to the logical user interface element of the value reference destination is updated using a concrete user interface element update function defined for the logical user interface element of the value reference destination. Including the step of updating.

[0093] Preferably, the logical user interface element includes a container type logical user interface element which may include another logical user interface element, and the step of performing processing for transmitting to the operation subject has generated the first message. If the logical user interface element is included in the container type logical user interface element including the logical user interface element that generated the first message, the first message is transmitted to the other logical user interface element; If not, transmitting a first message to the actor.

Preferably, the logical user interface element includes a container type logical user interface element which can include other logical user interface elements, and the user interface description document includes the container type logical user interface element and the container type logical user interface element. An inclusive relationship can be described with the logical user interface element included in the element, and the step of performing a process for transmitting to the operation subject includes, from the logical user interface element that generated the first message, the first message And sequentially propagating the first message upwards according to the hierarchical relationship of the container-type logical user interface elements that include the logical user interface element that generated the message.

[0095] Preferably, the step of performing processing for transmitting to the operation subject is a container type logical user interface element that is not included in another container type logical user interface element among the container type logical user interface elements including the logical user interface element. The method further includes transmitting, at the interface element, the first message to the operating entity in response to receiving the propagation of the first message.

Preferably, the logical user interface element includes a container type logical user interface element which may include another logical user interface element, and further, the second message is received at the first container type logical user interface element. Transmitting the second message to a logical user interface element included in the first container type logical user interface element in response to the first message.

Preferably, the first container-type logical user interface element includes a second container-type logical user interface element, and the second container-type logical user interface element further comprises a first container-type logical user interface element. In response to receiving the second message, updating the second message in a predetermined manner and propagating to a logical user interface element included in the second container type logical user interface element.

[0098]

DESCRIPTION OF THE PREFERRED EMBODIMENTS [Basic Concept] The present invention relates to a method and system capable of realizing a UI of an application program on an arbitrary device and providing the UI to a user by separating the UI from the application program. Therefore, first, the separation of the UI from the application program according to the first embodiment of the present invention will be described, and the components of the UI used in the present embodiment and the description of the UI using the UI will be described.

Referring to FIG. 1, Embodiment 1 of the present invention
Includes an application 1100 and a UI device 1200 for realizing a UI. Generally, the application 1100 and the UI device 1200 exist in different devices.

Referring to FIG. 1, application 1100 includes application program 1101 and UI used by application program main body 1101.
And a UI description document 1102 that describes That is, the application 1100 includes an application program main body 1101 and a UI description document 110 describing the UI thereof.
And two.

On the other hand, the UI device 1200 includes a UI realizing program 1201. This UI realization program 1200
Is a UI description document 110 in the application 1100.
2 is read, the UI presented there is realized by the method described later, and provided to the user. The UI realizing program 1201 uses this UI to communicate a message with the application program main body 1101 to transmit a user's instruction to the application program main body 1101 or to transmit the state of the application program main body 1101 and the instruction therefrom. And the like can be reflected on the UI by the UI realization program 1201.

FIG. 3 shows an example of the UI description document 1102. Referring to FIG. 3, in the UI description document 1102, information on components constituting the UI is described on a text basis in the first embodiment. The description based on the text has the effect of increasing the readability for humans and facilitating creation and maintenance. However, this UI
The description document 1102 does not necessarily have to be text-based, but may be in any form that is machine-readable and can be analyzed.

The components described in the UI description document 1102 are not specific components provided to the user by the specific UI device 1200 or the UI realizing program 1201, but each UI element in the interaction with the user. It is a component defined by a general and logical system by abstracting functions. In the present invention, such a component is referred to as a logical UI element. on the other hand,
A concrete UI component includes a specific UI component included in each specific UI device 1200 or the UI realization program 1201.
It is called an element.

The contents described in the UI description document shown in FIG. 3 will be described later. Referring to FIG. 1 again, the UI implementation program 1201 generates an image of each logical UI element based on the information of the logical UI element described in the read UI description document 1102. And the UI implementation program 12
01 implements a UI by assigning an appropriate concrete UI element to an image of each logical UI element. Here, the image is a management unit in the UI realization program 1201 relating to each logical UI element, and it is desirable that the image is realized as one object when programmed by object-oriented programming.

As described above, the logical UI element is a general-purpose logical UI element by abstracting all conceivable UI elements. In the present embodiment, the user performs an interaction with the UI element, that is, an interaction function provided by the UI element.
A classification hierarchy tree is generated based on the interaction function, and a logical U is created in correspondence with each node of each classification hierarchy tree.
The I element has been defined. FIG. 2 shows an example of a classification hierarchical tree of logical UI elements. In the classification hierarchical tree shown in FIG. 2, the higher the logical UI element is, the higher the abstraction level is, and the lower the hierarchical level is, the lower the abstraction level is. The highest node is called a root here.

Hereinafter, a typical logical UI element, entr
y, valuator, choice, outputComp and containerComp
Is described.

(Entry) entry is a UI element for inputting an arbitrary character string. Logical UI elements of lower abstraction derived from entry include textField and textAria as shown in FIG. The textField is a logical UI element that generically refers to UI elements used for inputting a relatively short character string, such as registration of a file name in a GUI. textArea
Is a logical UI element for generically referring to UI elements for inputting a character string extending over several lines.

(Valuator) The valuator is a UI element for setting an arbitrary continuous amount. derived from valuator,
Logical UI elements with a lower level of abstraction than valuator are slider, dial
including. slider is the slider used in the GUI,
This is a logical UI element that generically refers to a slide-type variable resistor used for AV (Audio Visual) equipment. dial is a logical UI element that generically refers to a dial-type variable resistor and the like.

(Choice) choice is a UI for selection operation.
Element. Logical UI elements derived from choice are unaryChoice, multiCh classified by the type of the selection operation.
oice and exChoice. unaryChoice tells the application program only the fact that an item was selected, multiChoice allows you to select any of multiple items, and exChoi
“ce” can select only one item from a plurality of options. The three logical UI elements of the choice system described above are further subdivided into several logical UI elements as shown in FIG.

(OutputComp) outputComp is a UI element used exclusively for outputting information, and is a Mete that outputs a continuous amount.
r, text that outputs an arbitrary character string, etc. are included as its lower elements. Although not shown in FIG. 2, UI elements for presenting any graphics, video, or audio may also be outputCo.
Defined as a sub-element of mp.

(ContainerComp) containercomp is a UI
This is a logical UI element for grouping other logical UI elements in a description document as one group. Such a logic U
The I element is hereinafter referred to as a container type logical UI element. Logical UI
It is convenient to use this container-type logical UI element when it is desired to handle the elements collectively in functional units. The container-type logical UI element can include the container-type logical UI element as its element. That is, the container-type logical UI element can constitute a nested UI.

As described above, the UI description document 1102 shown in FIG. 1 is described using the logical UI elements shown in FIG. The UI implementation program 1201 implements a UI provided to the user by associating each of the logical UI elements described in the read UI description document 1102 with a concrete UI element. For example, the slider shown in FIG. 2 is associated with a slider generally used in a GUI and a slide-type variable resistor used in AV equipment and the like. Radi derived from exChoice
oButton is associated with a radio button of a GUI and a radio button of hardware as used in a car radio.

Further, in the classification hierarchy shown in FIG. 2, it is possible to associate any concrete UI element with the logical UI element of the intermediate node. Since each hierarchy of the classification hierarchy tree represents the level of abstraction regarding the interaction function of the logical UI element, the logical UI element of any intermediate node is derived from the logical UI of the intermediate node lower in the classification hierarchy. It can be embodied by a concrete UI element associated with any of the elements.

By defining the above-described classification hierarchy systematically and implementing the UI according to the above-described rules, the feasibility of the UI provided by the application is guaranteed. That is, in the UI device 1200 or the UI realization program 1201 shown in FIG. 1, the logical U specified by the UI description document 1102 for some reason.
Even if the I element cannot be realized, it is a higher logical UI element that can be realized according to the classification hierarchy, or another lower logical UI derived from the higher logical UI element. Elements that can be realized may be used as alternative logical UI elements. The thus determined alternative logical UI element has some functional differences from the original logical UI element described in the UI description document 1102, but does not have a significant functional difference. As a result, if the alternative logical UI element is used, the UI device 12
Under the restriction of the UI elements that can be used in 00, a UI that reflects the intention of the creator of the application 1100 to the maximum can be provided.

The nature of the classification hierarchy as described above is based on the logic U expressed by the classification hierarchy as shown in FIG.
Increase the expandability of the I element system. As described above, the logical U
Since the feasibility of an I element is limited, as long as it is defined in the classification hierarchy tree, defining a new logical UI element in the classification hierarchy based on its interaction function makes it logical. The UI element system can be easily extended.

[Values and Value Candidates] A logical UI element can have one or more “values” representing the state of the logical UI element. Since the logical UI element is defined based on the interaction function, its value is determined based on the interaction with the user performed through the corresponding concrete UI element. Therefore, the method of updating the value depends on the type (definition) of the logical UI element. For example, if the logical UI element is an output-only logical UI element, its value represents the information that the logical UI element is presently presenting to the user. Logical U
If the I element accepts a user input, its value indicates information on the content input by the user.

A logical UI element can have a value as an initial value of the logical UI element. Further logical UI
Elements can have value candidates that are also used as choices. Since the method of updating the value differs depending on the type of the logical UI element, the details of how to use the value candidates also differ depending on the type of the logical UI element. For example, a logical UI below outputComp which is a logical UI element dedicated to output shown in FIG.
In the element, its value or value candidate represents the data output by this logical UI element and presented to the user. FIG. 2
Is used as an initial value in a logical UI element below entry which is a logical UI element for text input shown in FIG. Also, a logical UI element below the selection logical UI element choice
In an element, its value is used as its choice. That is, the value candidate is a value that is selectively used at an arbitrary timing inside the logical UI element.

It is desirable that the value candidate or value has another "name" in addition to the "value itself". For example, when the value itself is a long character string, it is convenient to treat the name as a symbol representing the value. Further, when different values included in different logical UI elements are given the same name, a relationship between the values having the same name can be defined.

[Attributes Related to Function] The logical UI element has a function attribute related to the interaction function. The function attribute is an attribute that can be optionally specified by a writer of the UI description document. A function attribute defined in a certain logical UI element is inherited by a lower logical UI element derived from the logical UI element in the classification hierarchy. For example, the continuous value input logic UI shown in FIG.
The element valuator is defined with upper and lower limit functional attributes for the values that can be set. And these functional attributes are derived from this valuator and the lower logical UI
Inherited by the slider and dial elements.

Further, the classification hierarchy and the function attribute are both defined based on the interaction function. Therefore, in the definition of a certain logical UI element, a function attribute inherited from a higher-level logical UI element may be fixed to an arbitrary attribute value. In this case, the creator of the UI description document cannot change the function attribute.

[Additional Attributes] In the present embodiment, the logical UI
Elements also have additional attributes that are not directly related. This additional attribute is mainly for defining an attribute directly given to the concrete UI element associated with the logical UI element. In the case of a concrete UI element using a GUI, information on rendering such as the color and shape of the concrete UI element can be given by the attribute. Hereinafter, such an attribute is referred to as an additional attribute.

[UI description document and UI description document definition information]
The UI description document 1102 (see FIGS. 1 and 3) is a document in which information on a logical UI element having the above-described properties is described. Information described in the UI description document includes:
Each logical UI element includes a name representing its type and a unique identifier in the UI description document. Further, the description information includes a description of information on a value candidate and a function attribute required depending on the type of the logical UI element. Further, an additional attribute of the logical UI element is described as needed.

The description format for the UI description document may be any format. However, in the present embodiment, as shown in FIG. 3, XML (Xt
ensible Markup Language). XML is a standard that defines the grammar of tagged documents. A document specified by XML is composed of a set of elements (nested structure) shown below.

Element description 1: <element name attribute specification> Content </ element name> Element description 2: <element name attribute specification> Element description 1 is a general description. In the element description 1, the content enclosed by the first <> is a start tag, and the portion enclosed by the second <> is an end tag. A meaning is added to the content enclosed by the start tag and the end tag by the element name and attribute specification described in the start tag and the end tag. In the element description 1, another element description may be included in the content. This allows a nested structure of multiple elements to be created.

The element description 2 is a description method in which the contents are not specified as compared with the element description 1.

In the present embodiment, the name of the logical UI element is used as the element name, and the identifier of the logical UI element and the function attribute are specified by attribute specification. Further, in the present embodiment, elements of value and property are defined for a value candidate and an additional attribute, respectively. Then, their names are described by specifying attributes, and the values or attribute values are described as contents. U
The value and the property in the I description document are described as the contents of the logical UI element. Furthermore, a container type logical UI
Regarding the element, by defining one or more logical UI elements in the contents of the panel with the element name of panel, it is defined so that the nested structure of logical UI elements, which is a feature of the container type logical UI element, can be expressed. I do.

In the present embodiment, the definition information of the description format as described above is described in the document definition information relating to the UI description document 1102. In the case of XML, since the definition information relating to the description format of the document is described in a description portion called DTD (Document Type Definition), the definition information of the UI description document is described in the DTD of XML also in the present embodiment. .

Referring to FIG. 1, UI implementation program 12
01 reads the UI description document 1102 and refers to the UI description document definition information included in the UI description document 1102,
It is verified whether the read UI description document 1102 is correctly described in light of this definition information. If the description is correct, the UI implementation program 1201 generates an image of each logical UI element. The image of the logical UI element includes, in addition to the information on the logical UI element described above, the value of the logical UI element.

FIG. 3 shows an example of the UI description document 1102 shown in FIG. FIG. 20 shows an example of a UI realized by reading the UI description document 1102 by the UI realization program 1201 shown in FIG.

Referring to FIG. 3, UI description document 1102
Is for viewing DVD software on a television receiver using a DVD (Digital Video Disc) player. The UI description document 1102 is represented by a nested structure of logical UI elements adopting a container-type logical UI element called an entire panel description 3000. The overall panel description 3000 includes a state panel description 3100 that displays the state of the DVD player or the television, a DVD player control panel description 3200, and a television control panel description 3300. These are shown in FIG.
State panel 20100, DVD control panel 20
200 and the TV control panel 20300.

The DVD player control panel description 3200 shown in FIG. 3 includes two container-type logical UI elements: a content selection panel description 3210 for selecting DVD content and a controller panel description 3220 for controlling DVD playback and the like. These correspond to the content selection panel 20210 and the controller panel 20220 shown in FIG. 20, respectively.

The TV control panel description 3300 shown in FIG.
Is a volume control panel description 3310 for controlling the volume of the television.
including. This description is based on the volume control panel 203 shown in FIG.
Corresponds to 10.

Referring to FIG. 3, the name attribute of each UI element is the identifier of the logical UI element. If the element is an additional attribute, its name attribute is the attribute name of the additional attribute. If the element is a value candidate, the name attribute is the name given to that value. If the element is a value candidate, the name attribute is not necessary. The function attribute of the logical UI element is described in a format for giving an attribute value based on the function attribute according to the attribute designation of each element as needed.

The logical UI elements, value candidates, and additional attributes used in the UI description document 1102 shown in FIG. 3 will be described below.

A logical UI element used in FIG. 3 includes a character output element label for outputting an arbitrary character string, a list selection element list for selecting an arbitrary plurality of items from a list, and one line. , A text input element textField for inputting text, a button element button for enabling button type selection input, and a slider element slider for inputting a continuous amount. Hereinafter, each will be described.

(Character output element label) Character output element label
Is an element that outputs its own value candidate. This part exists in the third, sixth, and ninth lines of the description 3100 in FIG. UI for generating the corresponding display screen of FIG. 20
In the device 1200, the identifier of each element (“DVDStat
e "," Contents "," TVVolume ").

(List selection element list) List selection element li
st is an element that gives the user his / her own value candidates as options and allows multiple selections among them. The content selection panel description 3210 shown in FIG. 3 describes a list selection element list having six value candidates. In the third line of this content selection panel description 3210,
There is a description about a functional attribute called spLine. This function attribute is an attribute for designating the number of value candidates presented to the user at one time. The description 3210 defines a value of 4 for this. If there are more value candidates than this number, the list is presented using a scroll bar or the like, as indicated by the identifier “contentsList” in the display 2010 of FIG.

(Text input element textField) textField
Is an element for one-line text input. The content selection panel 3210 shown in FIG. 3 describes a text input element textField having no value candidate. If the textField has a value candidate, the value candidate is presented to the user as an initial value. Content selection panel 2 shown in FIG.
A textField included in 0210 indicates a display example when user input has already been performed.

The content selection panel description 32 shown in FIG.
Reference numeral 10 describes information on three function attributes.
The columns attribute specifies the number of characters that can be entered in textField. Note that the remaining two function attributes valueRefs
And propDel will be described later.

(Button element button) Button element button
Is an element for transmitting only the fact to the outside when a predetermined operation is performed by the user. Therefore, the button element button does not have to have a value candidate. All buttons shown in FIG. 3 have no value candidates. FIG.
Each button indicated by 0 is displayed together with the identifier of the button element button. However, when the button element has a value candidate, the button may be displayed together with the value of the value candidate itself instead of the identifier of the button element button.

(Slider element slider) Slider element slider
er is an element for continuous quantity input. In this case, the value candidate for the slider element slider is used as its initial value.
The slider element slider included in the volume control panel 3310 shown in FIG. 3 describes information on two functional attributes, an upper attribute and a lower attribute. These function attributes specify the upper and lower limits of the specifiable continuous amount. These functional attributes are also inherited from the logical UI element valuator, which is an upper element of the slider element slider in the classification hierarchy shown in FIG. The slider element sl included in the volume control panel 3310 shown in FIG.
Each ider value candidate has a name. Not only for the slider element slider, but also for the value candidates of all the logical UI elements, the name given to it is additional information. These names are used for specifying a value candidate to be changed when the application program itself changes the value candidate. Update of the value candidate will be described later.

The additional attributes used in FIG.
There are three types: a color attribute, a flowOrientation attribute, and a visible attribute.

The color attribute specifies the value of a concrete UI element associated with a logical UI element. The attribute value of the color attribute is expressed by listing two hexadecimal digits (a total of six digits) for each value of RBG after "@".

The flowOrientation attribute is an additional attribute related to the display arrangement method of the concrete UI elements associated with the logical UI elements. The flowOrientation attribute is specified as an additional attribute of the container-type logical UI element. When this flowOrientation attribute is specified in a certain container-type logical UI element, the logical UI included in the container-type logical UI element
Elements are arranged according to the specification of the flowOrientation attribute. In FIG. 3, descriptions 3000, 3200, and 321
FlowOrientation at 0, 3300 and 3310
Vertical (vertical direction) is specified as an attribute. In other descriptions 3100, 3220 and 3311, hori
zontal (horizontal) is specified.

The visible attribute gives information on whether or not a concrete UI element associated with a logical UI element is presented to a user (whether or not to display it). If the attribute value of the visible attribute is "true", the concrete UI element is presented,
If "false", it is not presented.

(Message Communication) As shown in FIG.
The I realization program 1201 exchanges messages with the application program main body 1101. Hereinafter, U
A message transmitted from the I realization program 1201 to the application program 1101 is called an “upper message”, and the application program main body 1101
Is sent to the UI implementation program 1201 as a “downward message”.

The upward message is a message for notifying the application program main body 1101 of the operation performed by the user through the concrete UI element, that is, the state of the logical UI element. When receiving the upward message, the application program main body 1101 executes a process corresponding to the upward message. The logical UI element has a value representing the state in the UI realization program. The upper message includes an identifier of an arbitrary logical UI element and a value indicating the state of the logical UI element. The upward message is generated at an arbitrary timing and transmitted to the application program main body 1101. However, it is desirable that the upward message is generated and transmitted at the timing when the value of the corresponding logical UI element is updated, that is, at the timing when the user performs an input operation on the corresponding concrete UI element.

The down message is the UI realization program 1
Reference numeral 201 denotes a message used to update information or a state regarding an arbitrary logical UI element. The downward message is for updating the value, value candidate, function attribute, or additional attribute of the logical UI element to be updated.
Thus, the down message has a different structure depending on the type of change. Hereinafter, information required for each type will be described.

(Update of Value) The identifier of the target logical UI element and the value that the logical UI element should have, or the name of any value candidate that the logical UI element has, are included. In the former case, the UI implementation program 1201 changes the value of the corresponding logical UI element to the value specified in the message. In the latter case, the UI implementation program 1201 replaces the value candidate having the name specified in the message with the logical UI.
Take the value of the element.

(Update of Value Candidate) There are three types of messages related to updating of the value candidate of the logical UI element. The first type (value candidate update 1) is a type that changes the value itself of an arbitrary value candidate for an arbitrary logical UI element. Second
(Value candidate update 2) is a type for deleting an arbitrary value candidate for an arbitrary logical UI element. The third type (value candidate update 3) is a type in which a new value candidate is added to an arbitrary logical UI element. They are distinguished by the type included in these messages.

The value candidate update 1 includes the identifier of the target logical UI element, the name of the value candidate to be changed, and the value after the change. Upon receiving this message, the UI implementation program 1201 changes the value candidate of the specified logical UI element specified by the message to a value included in the message.

The message related to value candidate update 2 includes the identifier of the target logical UI element and the name of the value candidate to be deleted. When receiving the message belonging to the value candidate update 2, the UI implementation program 1201 deletes the value candidate of the logical UI element specified by the message specified by the message.

The message belonging to the value candidate update 3 includes the identification of the logical UI element to which the value candidate is added, the name of the logical UI element to be newly added, and the value itself corresponding to the name. . UI implementation program 1201
Receives a message belonging to value candidate update 3, and adds a value candidate having the name included in the message and the value itself included in the message to the logical UI element specified by the message.

(Update of Function Attribute) The message for updating the function attribute is newly set to the identifier of the target logical UI element, the name of the function attribute to be updated, and the function attribute. And attribute values. Upon receiving this message, the UI implementation program 1201 changes the attribute value of the function attribute of the logical UI element specified by this message specified by this message to the attribute value included in this message.

(Update of Additional Attribute) The message for updating the additional attribute is newly set in the identifier of the logical UI element to be updated, the name of the additional attribute to be updated, and this additional attribute. Attribute values. Upon receiving this message, the UI implementation program 1201 changes the attribute value of the additional attribute specified by this message of the logical UI element specified by this message to the attribute value included in this message.

As described above, the UI description document can be described in a format including a nested structure using container-type logical UI elements. When the logical UI element described in the UI description document has a nested structure based on the function,
The container-type logic U that forms the nest includes information on the identifiers contained in the upper message and the lower message.
It is convenient to include the identifier of the I element. For example, regarding the text input element textFileld included in the content selection panel description 3210 in FIG. 3, the identifier information included in the upper message and the lower message is changed to “DVD: con
It can be a list expression such as "tents: selectedContents". This representation includes the identifier of the nested container-type logical UI element.

Logical U contained in container type logical UI element
By expressing the identifier information of the I element in this way,
When the application program main body 1101 analyzes the upward message, it becomes easy to divide the cases based on the function of the UI. Also, the UI implementation program 12
When 01 analyzes the lower message, the search for the corresponding logical UI element specified by the lower message can be performed at high speed.

Next, a user interface realizing system and a user interface realizing method of the present invention will be described more specifically based on one embodiment of the present invention.

Referring to FIG. 4, the system according to the present embodiment includes a UI device (device 0) 4100 for providing a user with a UI for controlling a plurality of devices, and a plurality of devices. 4110-4160 and these UI devices 41
00 and a network 4000 such as IEEE 1394 that connects the plurality of devices 4110 to 4160 by bus.

FIG. 5 shows each device more specifically. This system includes a controller 5 corresponding to the UI device 4100.
100, a television receiver 5110 which is a specific example of the devices 4110 to 4160, a VCR (video cassette recorder) 5120, a CD-MD component stereo 5130, a personal computer 5140, and C
It includes an S (satellite broadcast) tuner 5150 and a network 5000 for connecting these to a network.

The controller 5100 communicates with the UI device 4100 and the network 5000 by wireless communication such as infrared rays.
It is realized in the style of connecting Controller 5100
Includes a UI device main body 5101 and a transmission / reception port 5102 connected to the network 5000 for communicating with the UI device main body 5101.

Referring to FIG. 6, UI device 5101 includes housing 6000 and display device 6 provided on housing 6000.
010, a cross button 6020 and buttons 6031 to 6034, which are input devices provided on the housing 6000.
And a transmission / reception port 6050 for realizing bidirectional communication with the transmission / reception port 5102.

A display panel 6010 graphically displays a controller panel for controlling an arbitrary device. It is preferable to use a touch panel display device as the display device 6010. In this case, the user displays the GUI (Graphical User Int) displayed in the display device 6010.
erface) It is possible to directly operate components with their own hands or a pen or the like. When the display device 6010 is not a touch panel type, the user can use the cursor button 60
20 and the buttons 6031 to 6034 can be used to perform an operation on the display. Even if the display device 6010 is a touch panel type, the touch panel may not be used for some reason.
The device 5101 can be operated.

Referring to FIG. 7, instead of UI device main body 5101 shown in FIG. 6, UI device 7000 having no display device is provided.
Can also be used. This type of UI device 7000
Is used together with a display device such as a television 7100, and information on an operation method using the controller body (UI device 7000) is displayed on a display device of a device such as a DV. Hereinafter, it is assumed that the display device is a television receiver, and the transmission / reception port 510 on the network side is used.
2 will be described assuming that the TV receiver 2 is built in the television receiver.

Referring to FIG. 7, the UI device main body 7 of this example will be described.
Reference numeral 000 includes a cross button 7020, buttons 7031 to 7034, and a dial 7040, which are input devices, and a transmission port 7050 for wirelessly transmitting operation contents of these input devices by the user to the transmission / reception port 5102. On the other hand, the television receiver 7100 includes a display screen 7200, and a reception port 7150 (an example of the transmission / reception port 5102) for receiving information wirelessly transmitted from the UI device main body 7000. Information necessary for controlling the UI device main body 7000 is displayed in a UI information area 7210 in the display screen 7200.

For example, in the UI information area 7210, FIG.
GUI similar to that displayed on the display device 6010 shown in FIG.
A part group may be displayed. Or UI information area 7210
Are devices that can be used on a network and input devices (buttons 7020, 7020, 7
031 to 7034 and the dial 7040).

In the former case, the cursor 7211 displayed in the UI information area 7210 can be moved by the cross button 7020 of the UI device main body 7000.
Then, the cursor 7211 is operated and input using the buttons 7031 to 7034 to which operations such as “determine” and “cancel” are assigned. In the latter case, the UI device body 70
Each input device provided with 00 is assigned in advance a function for defining an arbitrary device. User is UI
When an arbitrary input device provided in the device main body 7000 is operated, a corresponding function can be controlled. For example, when the volume control of the television 7100 is assigned to the dial 7040, the user
Is rotated by an arbitrary angle, control is performed such that the volume of television 7100 changes. In this case, although the function assigned to each button is unknown only with each button of the UI device main body 7000, the UI information area 7210
The function assigned to each input device can be known by referring to the information displayed in.

In the following description, a case will be described in which the UI device main body 5101 shown in FIG. 6 is used. In the following description, it is assumed that one of the devices shown in FIG. 4 operates as a server device that manages devices connected to the network 4000.
If the UI device controls only an arbitrary device on a one-to-one basis, such a server device is unnecessary. However, as in the present embodiment, UIs for a plurality of devices are dynamically generated to
If operation can be performed with one UI device, it is better to provide a server device on the network. In FIG. 4, the personal computer 4140 (the personal computer 5140 in FIG. 5) or the UI device main body 4100 (51)
00) is preferably responsible for the functions of such a server device.

(UI Apparatus) Referring to FIG.
100 includes a UI device 8100 and a control target 8200. Communication between the UI device 8100 and the control target 8200 realizes target device control. The control target 8200 is an arbitrary device. However, if the network is managed by a server device, U
The communication partner of the I device 8100 is directly a server device, and the server device communicates with the control target device to control the device.

Referring to FIG. 8, UI device 8100 transmits an arbitrary UI description document and a downward message to control object 820.
0, a transmission unit 8100 for transmitting an upward message to the control target 8200, a UI management unit 8101 for managing message transmission / reception and overall behavior of the UI device 8100, and a UI device. 8100, a logical UI element management unit 8102 for generating and managing an image of a logical UI element;
A concrete UI element management unit 8103 for managing concrete UI elements provided directly to a user by the user 100, an output device 8105 for providing a UI to the user, and an input device for receiving a user's input operation using the UI 8106, and an input / output management unit 8104 for managing input / output of the output device 8105 and the input device 8106.

Referring to FIGS. 10 and 11, a description will be given of a method for realizing a user interface according to an embodiment of the present invention in UI device 8100. FIG. In the user interface realizing method of the present embodiment, first, in step S101 of FIG. 10, an arbitrary UI description document is read and analyzed. This is the UI management unit 810 shown in FIG.
1 executes. The UI description document is received from the control target 8200 via the receiving unit 8107. UI management unit 8101
Refers to the UI description document definition information 8109 and confirms that the received UI description document is described in a predetermined description format. If an error is found in the UI description document, a UI description document notifying the fact is generated, and the same processing is performed.

If no error has occurred, then S1
In 02, a UI is provided to a user by associating a concrete UI element with a logical UI element described in a UI description document. FIG. 11 shows the details of step S102.

Referring to FIG. 11, first, at step S201
Is associated with a logical UI element and a concrete UI element. In step S201, the UI management unit 8101 in FIG.
This is performed with reference to the element correspondence table 8110. FIG. 9 shows an example of the UI element correspondence table.

In the UI element correspondence table shown in FIG.
The information on the element includes a name representing the type of the logical UI element and, if necessary, attribute information describing an attribute value on a functional attribute. The information on the concrete UI element includes a concrete UI element that can be realized by the UI device and parameters (attributes) necessary for the concrete UI element.

Referring again to FIG. 11, in S201, U
With respect to all the logical UI elements described in the I description document, an appropriate correspondence among the correspondences described in the UI element correspondence table of FIG. 9 is searched.

This search is performed as follows. First, a search is made for a correspondence relationship in which both match perfectly and have a feasible concrete UI element. If the search is successful, the correspondence is used as the search result. If the search fails, a search is made for a correspondence that matches only under the name of the logical UI element and that has a feasible corresponding concrete UI element. If this search also fails, an alternative logical UI element is determined with reference to the classification hierarchy of the logical UI elements shown in FIG. Search results. The substitute logical UI element is a logical element that is described in the UI element correspondence table among the logical UI elements higher than the target logical UI element, and the logical element closest to the logical UI element limited in the classification hierarchy is the UI element. is there.

Referring to FIG. 11, S201 performs a process of searching for a correspondence relationship as described above and associating a concrete UI element with a logical UI element or an alternative logical UI element described in a UI description document.

Subsequently, in S202, an image of the corresponding logical UI element is generated and initialized based on the search result performed in S201. The initialization of the logical UI element is mainly processing for initializing the value of the logical UI element. In step S202, the logical UI element management unit 81
02 refers to the logical UI element library 8111. After that, the management of the logical UI element is performed by the logical UI element management unit 8102.

Referring to FIG. 11, in S203, S20
Based on the search result of 1, the corresponding concrete UI element is initialized. This initialization is based on the search result in S201, based on the attribute information on the concrete UI element and the information on the corresponding logical UI element described in FIG. 9, that is, the above-described values, value candidates, function attributes, and load attributes. It is performed by

Referring to FIG. 11, in S203, concrete U
I element management unit 8103 or logical UI element management unit 810
2 is performed. Which of the two processes is performed is determined by the logical UI element library 8111 and the concrete UI.
It is determined by the contents of the element library 8112. If the logical UI element library 8111 has knowledge about the parameters of the concrete UI elements, this processing is performed in the logical UI element management unit 8102. The concrete UI element library 8112 is a logical UI element value candidate,
This process is performed in the concrete UI element management unit 8103 if the information has the function attribute and the load attribute. In any case, the initialized concrete UI element is
Thereafter, it is managed by the concrete UI element management unit 8103.

Referring to FIG. 11, in S204,
The UI is presented to the user using the concrete UI element initialized in S203. When S204 ends, step S102 ends. Referring to FIG.
I is provided to the user by the output device 8105 via the input / output management unit 8104.

Referring again to FIG. 10, after S102, S1 is executed.
03, the input / output management unit 8104 causes the input device 80
It is determined whether or not the user's operation input has been made to the number 6. When it is determined that there is a user operation input, the control proceeds to S105 in FIG. Otherwise, control proceeds to S104.

At S104, UI management section 8101 determines whether or not a downward message has been received from control object 8200 via receiving section 8107. If it is determined that a downward message has been received, the control proceeds to S109; otherwise, the control returns to S103. S103 and S104 described above may be executed individually using multi-thread programming.

In S105, the concrete UI element management unit 810
In 3, a concrete UI element that has been operated by the user is searched for. When there is no corresponding concrete logical UI element, the control returns to S103, and when there is a corresponding concrete logical UI element, the control proceeds to S106.

In S106, it is determined whether or not it is time to update the value of the logical UI element associated with the concrete UI element searched in S105.
If it is not the update timing, the control returns to S103; otherwise, the control proceeds to S107. The timing for updating the value of the logical UI element differs depending on the type of the logical UI element. For example, the timing of updating the value of the entry-type logical UI element shown in FIG. 2 is basically when a character string is input. In the choice-based logical UI element, the update timing is when the option is selected by the user.

If it is time to update the logical UI element, S
If determined by 106, the value of the logical UI element whose value is to be updated is changed, and an upward message is generated using the identifier of the logical UI element and the updated value. Both steps of S106 and S107 are the logical U of FIG.
This is performed by the I element management unit 8102.

Further, in S108, the upward message generated in S107 is transmitted by the UI management unit 8101 to the control target 8200 through the transmission unit 8108. After transmitting this upper message, the control is S
Return to 103.

When it is determined in S104 that a downward message has been received, the control proceeds to S109. S
In 109, the logical UI management unit 8102 shown in FIG. 8 searches for a logical UI element targeted for a downward message. Here, if the target logical UI element does not exist, the control returns to S103, and if there is, the control proceeds to S110.

In S110, the logical UI element management unit 810
2, the logical U that is the object of the downward message
For the I element, the logical UI element is updated according to an update method for each type of downward message. And the logical UI
The concrete UI element associated with the element is updated according to the update content of the logical UI element.

(Arbitrary Device) Referring to FIG. 12, each device includes a device main body 12004, a control program 12001 for controlling device main body 12004, and a UI description document describing a UI separated from the control program. 1200
3, a management unit 12005 for managing the control program and communication with the outside, and a transmission unit 12006 and a reception unit 12007 for performing communication with the outside. As shown in FIG. 12, in the device according to the present embodiment, a UI description document 12003 is stored in a control program 1203 which is an application program for controlling the device main body 12004.
It is characterized in that it is separated from 01. Each device is
It further includes UI description document definition information 12002 which defines the description format used in the UI description document 12003. This UI description document definition information 12002 does not necessarily need to exist, but when the control program 12001 dynamically generates or changes its own UI description document 12003, the UI description document definition information 12002 is necessary. is there.

The management unit 12005 reads the UI description document 12003 at an arbitrary timing, and transmits the read UI description document 12003 to the server device or the UI device via the transmission unit 12006. Although the transmission timing is arbitrary, it is preferable that the transmission be performed when the device becomes available on the network, or when a transmission request of a UI description document is received from another device. Also, the management unit 12005 sends an upward message from the UI device or the server device received by the reception unit 12007 to the control program 12001. The management unit 12005 further has a function of transmitting the downward message transmitted from the control program 12001 to the UI device or the server device via the transmission unit 12006.

The control program 12001 includes the management unit 12
The main unit 12004 is controlled based on the upper message sent from 005. Control program 1200
1 generates a downward message and sends it to the management unit 12005 to change the UI provided to the user by the UI device. The upper message and the lower message are as described above.

(Server Device) The server device manages each device connected to the network together with each UI description document. The server device also provides a UI description document of an arbitrary device to the UI device, and relays a message exchange between the device and the UI device.

The UI description document provided to the UI device is not only a UI description document of an arbitrary single device but also a UI description document generated by synthesizing a UI description document of a plurality of devices and / or applications. Good.

Referring to FIG. 13, the server device manages the devices connected to the network and the UI description document thereof, and also serves as a bridge for message communication between the UI device and an arbitrary device. A management unit 13001,
A transmission unit 13006 and a reception unit 13007 for communicating with another device, and UI description document definition information 13002
, A UI description document generation / modification unit 13005 that generates and modifies a UI description document 13003 provided by the server device to the UI device, and manages devices connected to the network and the UI description document. Connection device information 13004 for connection.

The UI management unit 13001 receives the UI description document of another device connected to the network,
07 at an arbitrary timing. The UI management unit 13001 stores the acquired UI description document in the connected device information 13004 together with network information such as the address of each device on the network. Further, the UI management unit 13001 requests the UI description document generation / correction unit 13005 to generate or modify a UI description document to be provided to the UI device at an arbitrary timing,
The corrected / generated UI description document is transmitted to the UI device via the transmission unit 13006.

Upon receiving the upward message from the UI device through the receiving unit 13007, the UI management unit 13001 corrects the upward message by reflecting the generation / correction of the UI description document by the UI description document generation / correction unit 13005. The corrected upper message is transmitted to the device to be controlled through the transmission unit 13006. When a downward message is received from an arbitrary device through the receiving unit 13007,
The downward message is corrected to reflect the generation / correction of the UI description document in the UI description document generation / correction unit 13005,
The corrected downward message is transmitted to the UI device via the transmission unit 13006.

(Image of Logical UI Element) The image of the logical UI element managed by the logical UI element management unit 8100 has been described with reference to FIG. Hereinafter, the configuration of an object and an arbitrary processing procedure when a logical UI element is treated as an object in object-oriented programming will be described. The functions and properties of the logical UI elements which have not been described above will be described. Hereinafter, some functions / properties will be newly described first, and then the operation of the logical UI element will be described.

With respect to the upper message and the lower message, the identification information contained in the upper message and the lower message is transmitted to the container type logical UI.
The method of expressing as an identifier list reflecting the nested structure of elements has already been described. In the system according to the present embodiment, the upper message and / or the lower message having the identifier information expressed as the identifier list are converted into the corresponding logical UI element or container type logical U.
Propagate and process between objects of the I element. Therefore, in order to perform message propagation processing, the container type logical UI element has an upper message propagation function and a lower message propagation function.

First, the generation expression of the upper message propagation function and the lower message propagation function of the container type logical UI element will be described. In the following description, in the present embodiment, an arbitrary logical UI element (including a container type) holds a reference destination to a parent element having a nested structure, and the container type logical UI element has the container type logical UI element. It is assumed that the UI element holds a reference destination to a child element having a nested structure formed by the UI element.

First, the upward message propagation function will be described. When receiving the upward message from the child element in the nested structure, the container-type logical UI element updates the upward message by adding its own identifier to the identifier information included in the upward message. And this container type logic U
The I element propagates the updated upward message to its parent element. Here, this container type logical UI
If the element does not have a parent element, that is, if this container-type logical UI element is
If the element is an I element, this container-type logical UI element transmits the upward message updated as described above to the operation target 8200 (see FIG. 8). In this case, the propagated message itself is the upper message. Logical UI elements that do not form a nested structure perform the same message propagation as the logical UI element that is the highest level of the nest.

[0202] The downward message propagation function is as follows. When receiving the downward message from the parent element in the nested structure, the container-type logical UI element determines whether or not the identification information included in the downward message includes its own identifier. If the identification information included in the received downward message does not include its own identifier, the downward message is irrelevant to itself and the logical UI element included therein, and no further processing is performed. On the other hand, when the identification information included in the received downward message includes its own identifier, the container-type logical UI element modifies the downward message by deleting its own identifier from the identifier information. Then, the updated downward message is propagated to all of its own child elements. Thus the down message is
If the identifier is included in the identifier information included in the received downward message and is transmitted to the child element in order, the child element determines that it is a downward message addressed to itself.

When a downward message is propagated to a child element as described above, it is not necessary to propagate the message to all child elements.
By examining the identifiers included in the identifier list of the downward message, it is possible to search for a child element to propagate the downward message. Therefore, the downward message may be propagated only to the child element thus searched.

As described above, a message propagates in a nested structure. Hereinafter, propagation of an upper message is referred to as upper propagation, and propagation of a lower message is referred to as lower propagation.

(Message Transmission Right) In this method of realizing the user interface, it has been described that when the user performs an input operation on the UI device 8100, all the contents are transmitted to the control target 8100 as an upper message. Was. However, in practice, some of the logical UI elements do not require immediacy as the timing at which the upper message should be generated, or some may be convenient to transmit together with the upper message of another logical UI element. In such a case, it is preferable to transmit the upper messages of several logical UI elements collectively, rather than transmitting the upper messages individually by a plurality of logical UI elements.

Therefore, in the present embodiment, the above-described problem is actually addressed by introducing the concept of message transmission right with respect to transmission of an upward message. The right to send a message can also be said to be the right to send an upward message from itself. Basically, all the logical UI elements have the right to send a message, and usually use this right to send an upper message to the outside. The term “outside” here means a container-type logical UI element that includes the logical UI element if the above-described message propagation is adopted. On the other hand, this message transmission right can be delegated to another logical UI element. A logical UI element that has delegated its message transmission right to another logical UI element cannot transmit an upward message to the outside. On the other hand, the logical UI element to which the message transmission right has been transferred (the logical UI element of the transfer destination) transmits the upper message of both its own logical UI element and the logical UI element of the transfer source to its own logical UI element. The data is transmitted to the outside collectively at the transmission timing of the UI element.

To specify whether to transfer the message transmission right of a logical UI element to another logical UI element, a logical U
A delegation destination attribute is defined as one of the functional attributes of the I element.
If the transfer destination attribute is defined as the information on the logical UI element described in the UI description document, the message transmission right of the logical UI element is transferred to the logical UI element specified by the definition. When the delegation destination attribute is not specified, it means that the logical UI element holds the message transmission right.

(See Value) According to the description so far, a message that is not directly used in the operation target 8200 in FIG. 8 may have to be transmitted as an upper message. For example, when a value update in an arbitrary logical UI element is used only to update another logical UI element. For example, UI according to user operation
Is dynamically changed. In this case, there is no need to send this upward message to the application program itself. In such a case, if a message is exchanged between the control target 8200 and the application program body one by one, network traffic will be unnecessarily increased. Unnecessary load is also applied to the operation target 8200.

In order to deal with such a problem, the system according to the present embodiment introduces a relationship called a value reference relationship between arbitrary logical UI elements. The value reference refers to a relationship between logical UI elements such that an arbitrary logical UI element can update its own value by referring to the value of another logical UI element. When an arbitrary logical UI element specifies the logical UI element of the value reference destination, the logical UI element of the value reference destination provides a value to the reference source logical UI element at an arbitrary timing. However, it is preferable that the arbitrary timing is, for example, a timing at which the value of the logical UI element of the value reference destination is updated.

When the reference logical UI element receives this value from the reference logical UI element, it updates its own value by applying the provided value to the value reference processing model specified in advance. As the value reference processing model, a model in which a provided value is set as its own value, or a model in which an arbitrary function is defined and its own value is obtained by the function is considered.

In order to specify the logical UI element of the value reference destination, a value reference destination attribute is defined as one of the functional attributes of the logical UI element, and an attribute for specifying the name and / or parameter of the value reference model is provided. These attributes are given as information on the logical UI elements described in the UI description document. If the value reference destination attribute is not specified, it means that the logical UI element does not perform value reference.

The value reference processing model may be described using an arbitrary script language or the like. When the UI description document is described in XML, any model can be easily realized because XML also supports a script language.

(Operation of Logical UI Element Object) The logical UI element object having the above-described framework of the message transmission right and the value reference will be described in detail with reference to the drawings. FIG. 14 shows the configuration of a logical UI element object managed by the logical UI element management unit 8102. Referring to FIG. 14, the logical UI element object includes an object function group 14100 and object data 14200. Object function group 14
100 includes one or a plurality of functions, and each function updates corresponding data in the object data 14200. The object data 14200
Data in the object function group 14100
Cannot be updated without using the functions included in.
By doing so, the independence of the logical UI element object from the outside can be enhanced, and it can be commonly used by various applications.

The object function group 14100 is a logical U
A logical UI element generation function 14101 for generating and initializing the I element object itself, a concrete UI element update function 14105 for initializing or updating a concrete UI element associated with the logical UI element, and a concrete UI element Object data 142 based on a user operation on
A value update function 14102 for updating the value in 00,
A message generating function 14103 for generating an upper message transmitted to the control target 8200 shown in FIG. 8 and propagating to the outside, and a lower message transmitted from the control target 8200 shown in FIG. 8 and propagated through a predetermined route Based on the logical UI element update unit 14104 for updating the data in the object data 14200, and the object data 1420 by referring to the values of other logical UI elements.
A value reference function 14106 for updating a value within 0.

The operation of the logical UI element object shown in FIG. 14 will be described with reference to FIGS. Note that the flowchart shown in FIG. 15 shows a process corresponding to, for example, S107 shown in FIG.

Before starting this processing, the concrete UI shown in FIG.
The element management unit 8103 is assumed to determine the specific UI element operated by the user in S105 illustrated in FIG. 10 and transmit the processing content of the determined specific UI element to the corresponding logical UI element. Referring to FIG. 14, value update function 14102 receives this processing content, and determines in S106 shown in FIG. 10 whether or not it is time to update the value of the logical UI element. If it is determined that it is time to update the value of the logical UI element, the value update function 14
102 is the process of S107 shown in FIG.
5 is executed.

Referring to FIG. 15, in S301, the value included in object data 14200 is updated. Subsequently, in S302, an upward message is generated and propagated upward. Subsequently, in S303, it is determined whether or not there is a reference source logical UI element which is to refer to the value of this logical UI element. Is performed.

The reference destination logical UI element of the value reference is registered as a reference destination logical UI element in the reference source logical UI element that references the value. In the system of this embodiment, when the logical UI element object is initialized in the logical UI element management unit 8102 shown in FIG.
Check the value reference logical UI element of the element. Then, a reference source logical UI element is registered as object data 14200 shown in FIG. 14 for each logical UI element to be a value reference destination. The processing of S303 and S304 is performed by referring to this information.

With reference to FIG. 16, the processing in S301 of FIG. 15 will be described. First, in step S401, the value of the object data 14200 shown in FIG. 14 is updated based on the operation performed on the concrete UI element. S40
In 2, it is determined whether the concrete UI element needs to be updated. If it is determined that the UI element needs to be updated, the concrete UI element is updated in S403. The updating of the concrete UI element in this case is necessary, for example, when the value of the logical UI element is reflected on the outer shape of the GUI, particularly when the GUI is used.

Subsequently, referring to FIG. 17, S shown in FIG.
Details of the processing of 302 will be described. First, in S501, it is determined whether or not there is a transmission right transfer source logical UI element that transfers the transmission right of an upward message to this logical UI element. If the transmission right transfer source logical UI element exists, in S502, an upper message of the transfer right transfer source logical UI element is acquired. As mentioned above, if the right to send upstream messages is delegated,
In the logical UI element of the transfer source, the logical UI element of the transfer destination is registered. In the present embodiment, the logic U shown in FIG.
When the logical UI element object is initialized in the I element management unit 8102, the transfer right logical UI elements registered in this manner are scanned to confirm the transmission right transfer destination logical UI elements of all the logical UI elements. I do. Then, each of the logical UI elements to which the transmission right is transferred is shown in FIG.
In the object data 14200 shown in FIG. 4, information for specifying the logical UI element of the transfer source is registered. The processing of S501 and S502 is performed by referring to this information.

Next, in step S503, an upward message to be transmitted is generated from this logical UI element. The upper message generated here is a message including the identifier and the value of the logical UI element. In S502, another logical UI
If the upper message of the element (transfer source logical UI element) is obtained, the upper message is concatenated.

Further, in S504, it is determined whether or not it has the transmission right of the upward message. If the user has the transmission right, the upward message is propagated upward in S505. S
If it is determined in 504 that the user does not have the transmission right, the upward message generated in S503 is stored in the object data 14200 shown in FIG. By storing the upward message in the object data 14200 in this way, the upward message can be acquired when the transmission right transfer destination logical UI element is required.

The processing of FIGS. 15 to 17 described above corresponds to the value providing function 14102, the message generating function 14103, and the value providing function 14106 shown in FIG.
107 is performed.

Next, referring to FIG. 18, the S shown in FIG.
Details of 110 will be described. Before starting this processing, the concrete UI element management unit 8103 shown in FIG.
A logical UI element targeted for a downward message is searched for in S109 shown in FIG. As described above, in this embodiment, downward messages are propagated downward. Therefore, this downward propagation becomes S109 shown in FIG. Furthermore, the logical U of the lowest layer in the nested structure of the logical UI element
In the I element, the processing of S110 in FIG. 10, that is, the processing whose details are shown in FIG. 18 is executed.

Referring to FIG. 18, in S601, appropriate data in object data 14200 shown in FIG. 14 is updated in accordance with the processing method of the downward message described above. Subsequently, in S602, it is determined whether the corresponding concrete UI element needs to be updated based on the data update performed in S601. If there is a need for updating, the concrete UI element is updated in S603.

In either case, control proceeds to S604. S
At 604, it is determined whether or not the value has been changed. If there is no change in the value, the process ends. If the value has been changed, the process proceeds to S605.

In S605, based on the updated value,
The transmission message in the object data 14200 shown in FIG. 14 is updated. The processing performed in S605 is the same as the processing performed in S503 shown in FIG.

As in steps S303 and S304 shown in FIG. 15, in step S606, it is determined whether or not there is a reference logical UI element that refers to the value of the logical UI element.
If the value reference source exists, the logical U of the reference source is determined in S607.
Provide this value for the I element. This is the end of the process.

The processing shown in FIG. 18 is performed by the logical UI element updating function 14104 and the concrete UI element updating function 14105 shown in FIG.

Next, the operation of the value reference processing 14108 of the value reference function 14106 shown in FIG. 14 will be described with reference to FIG. The value reference process is executed in the reference logical UI element object when a value is provided from the value reference destination logical UI element object to the value reference source logical UI element object. Logical U of value reference destination
The value provision from the I element object is performed in S3 shown in FIG.
04 and S607 shown in FIG.

First, in S701, it is confirmed that a value has been provided from a value reference destination. Subsequently, in step S702, the value of this logical UI element object is calculated using a predetermined value reference processing model.
Update the appropriate value in 00 (see FIG. 14).

Next, in step S704, an upward message is generated and propagated using the message generating function 14103 shown in FIG. 14 according to the flowchart in FIG. However,
When a value is provided from the logical UI element of the value reference destination in S607 shown in FIG.
When the value of the I element is updated according to the downward message and the value is provided according to the result, it is preferable that only the upward message is generated in S503 in the processing shown in FIG. 17 and the message is not propagated. . Subsequently, in S705, it is determined whether or not it is necessary to update the concrete UI element by the concrete UI element update function 14105 shown in FIG. If it is determined that the concrete UI element needs to be updated, the concrete UI element is updated in S706, and the process ends.

The embodiments disclosed this time are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

[Brief description of the drawings]

FIG. 1 is a diagram showing a basic concept of an embodiment of the present invention.

FIG. 2 is a diagram illustrating a classification hierarchy tree of logical user interface elements.

FIG. 3 is a diagram illustrating an example of a user interface description document.

FIG. 4 is a diagram schematically illustrating a network system according to an embodiment of the present invention.

FIG. 5 is a diagram more specifically showing a network system according to an embodiment of the present invention.

FIG. 6 is an external view of an example of a user interface device used in the system according to the embodiment of the present invention.

FIG. 7 is an external view of another example of the user interface device used in the system according to the embodiment of the present invention.

FIG. 8 is a block diagram of a user interface device.

FIG. 9 is a diagram illustrating an example of a UI element correspondence table.

FIG. 10 is a flowchart illustrating a method for implementing a user interface according to an embodiment of the present invention.

FIG. 11 is a flowchart illustrating a method for implementing a user interface according to an embodiment of the present invention.

FIG. 12 is a block diagram of each device.

FIG. 13 is a block diagram of a server device.

FIG. 14 is a diagram illustrating the configuration of a logical UI element object managed by a logical UI element management unit 8102.

FIG. 15 is a flowchart illustrating an operation of a logical UI element object.

FIG. 16 is a flowchart illustrating an operation of a logical UI element object.

FIG. 17 is a flowchart illustrating an operation of a logical UI element object.

FIG. 18 is a flowchart illustrating an operation of a logical UI element object.

FIG. 19 is a flowchart illustrating an operation of a logical UI element object.

FIG. 20 is a diagram illustrating an example of a user interface.

[Explanation of symbols]

1100 application, 1101 application program main body, 1102 user interface description document, 1200, 5101, 7000 user interface device, 1201 user interface realization program.

Continued on the front page (72) Inventor Naoki Urano 22-22 Nagaikecho, Abeno-ku, Osaka-shi, Osaka F-term (reference) 5B069 AA01 CA02 GA03 JA01 JA06 5K032 AA03 BA01 CC01 DA01 DB22

Claims (14)

[Claims] 1. A user interface element that constitutes a user interface between a certain operation subject and a user is described using a logical user interface element that abstracts the function of the user interface element in the interaction with the user. Reading and analyzing a user interface description document, and referring to the read user interface description document, using a concrete user interface element corresponding to the described logical user interface element, a user interface between the actor and the user And a process for generating a first message in a predetermined first format in response to a user operation on a concrete user interface element included in the user interface and transmitting the message to the operation subject Do And a step, the user interface implementation in an apparatus for providing a user interface to the user. 2. The method according to claim 1, further comprising: receiving a second message in a predetermined second format from the operation subject; and changing the user interface in response to the received second message. The user interface realizing method according to claim 1, comprising: 3. The user interface description document includes:
It is possible to define a concrete user interface element corresponding to a logical user interface element, and the step of modifying the user interface comprises: responding to the received second message,
The user interface realizing method according to claim 2, further comprising: updating a logical user interface element that has received the above message; and updating a concrete user interface element corresponding to the updated logical user interface element. 4. The user interface description document includes:
The user interface realizing method according to claim 1 or 2, wherein the document is a document describing information on one or more logical user interface elements. 5. The logical user interface element,
5. The method according to claim 1, wherein each of the functions is defined to form a classified hierarchical tree that is abstracted and hierarchized at a plurality of levels of abstraction in accordance with the function of each function and the degree of abstraction of the function. The method for realizing a user interface described in Crab. 6. The description of the logical user interface element of the user interface description document, the description defining an identifier for distinguishing itself from other logical user interface elements in the user interface description document; A description defining a value representing a state of the user interface element, further comprising: providing the corresponding concrete user interface element with the corresponding identifier; and providing the concrete user interface element with a corresponding value. A method for realizing a user interface according to claim 5. 7. The description of the logical user interface element further defines a name assigned to the value and the value candidate, respectively, further comprising: a value and a value candidate given to the concrete user interface element. 7. The method according to claim 6, further comprising the step of assigning a corresponding name to the user interface. 8. A first attribute is defined for a logical user interface element, wherein the first attribute is inherited by a lower logical user interface element derived from the logical user interface element in the classification hierarchy tree. The user interface realizing method according to any one of claims 1 to 7, which is performed. 9. The logical user interface element,
The user interface realizing method according to claim 1, further comprising a second attribute independent of a function of the logical user interface element. 10. The logical user interface element according to claim 1, wherein at least one of the logical user interface elements is a container-type logical user interface element that can include one or more other logical user interface elements. The method for realizing a user interface described in Crab. 11. A user in which the user interface description document describes a definition relating to a description format of the user interface description document, information relating to the logical user interface element and a definition relating to the description format, and information relating to the classification hierarchy tree. 6. The user interface implementing method according to claim 5, wherein the method is based on interface description document definition information. 12. A container-based logical user interface element, wherein one logical user interface element is a container-type logical user interface element that can include one or more other logical user interface elements. A logical user interface element for identifying a logical user interface element, wherein the identifier is an identifier of a logical user interface element in the container type logical user interface element;
An identifier list arranged according to a hierarchical structure in the container-type logical user interface element, wherein the step of receiving the second message includes: receiving the second message and including the second message in the received second message. Determining whether the identifier list included in the received second message does not include its own identifier, and processing in response to the determination that the identifier list included in the received second message does not include the own identifier. Ending the step of deleting the own identifier from the identifier list in response to the identifier list included in the received second message being determined to include the own identifier; Determining whether or not the identifier list after deleting the identifiers is empty. Updating the self-identifier according to the content of the received second message in response to the identifier list having been deleted being determined to be empty. The method further comprising, in response to the identifier list after removing its own identifier being determined to be non-empty, including the identifier list after removing its own identifier.
3. The method according to claim 2, further comprising the step of transmitting the message to other logical user interface elements included therein. 13. For each of the logical user interface elements, a user interface description document may include:
It is defined whether the user has a message transmission right that is a right to transmit the first message to a predetermined transmission destination or is delegated to another, and the step of performing a process for transmitting the first message to the operation subject is: Generating the first message in response to a user operation on a concrete user interface element corresponding to the logical user interface element included in the user interface; and whether the logical user interface element has a message transmission right. Determining whether to send the first message to the predetermined destination according to whether it is determined that the logical user interface element has the right to send a message. The method for realizing a user interface according to claim 1, comprising: 14. An attribute relating to a certain logical user interface element can be defined in the user interface description document, and the attribute is a value that the logical user interface element has, and the other logical user interface element that is a value reference source And a value reference for updating its own value by applying a predetermined value reference model to a value provided from another logical user interface element to which the value is referred to. 2. The user interface realizing method according to claim 1, further comprising having a processing function.