JPH01229321A - User interface generation tool - Google Patents

Abstract

PURPOSE:To easily constitute a natural language interface and the like through the use of a dictionary by generating and executing a command for the input of the natural language in accordance with a dictionary generated by a prescribed defined system. CONSTITUTION:A display system processing means 14 automatically registers and controls information on all objects including parts objects to be displayed in an access table 21. Thus, an input system processing means 17 collates the table, whereby all the displayed objects can be recognized by the input from the pointing device of a mouse and the like. When an analysis dictionary 13 is generated in accordance with the prescribed system, a processor automatically converts the natural language into an object operation language by the means 17 in accordance with the registered information. Consequently, the natural language interface of Japanese and the like can easily be generated.

Description

[Detailed description of the invention] [overview〕 In object-oriented data processing systems.

This invention relates to a user interface creation tool that supports the creation of user interfaces for displaying and manipulating the contents of objects.

Visual improvements by managing the display area of objects
The purpose of this paper is to provide a means for easily and uniformly constructing a direct graph ink interface and a natural language interface using a dictionary.

In response to an instruction to display an object with predetermined definition information,
A display system processing means that displays the contents of the object and part object and also registers the display object and its occupied area information in the access table, and a lower stage that extracts the object according to the access table in response to input on the object display screen. and an input system processing means having a means for performing menu processing for the method and a means for generating and executing a command according to a dictionary for natural language human power.

[Industrial application field]

The present invention relates to a user interface creation tool that supports creation of a user interface related to object content display and operation in an object-oriented data processing system.

Recently, objects and data that appear in a given problem are managed as objects with attribute values and operation procedures (methods), and messages are sent and received between objects.
Object-oriented processing systems that process problems have been put into practical use and are in use. In such a system, a means is desired that allows the user to easily construct a desired processing environment.

[Conventional technology and its problems]

The user interface is, so to speak, a part related to the exchange of information between humans and computers, and its operability is greatly affected by its contents. Traditionally, user interfaces are
Depending on each problem.

Everything was designed independently by the developer.

Therefore, conventional user interfaces have many machine-dependent and application-dependent parts, and require a large number of man-hours to create a user interface in order to provide a user interface with good operability. Furthermore, since it is not machine independent, it is difficult to reuse (port) it in other systems.

In addition, conventionally, various interfaces such as so-called menu-based interfaces, command-based interfaces, and natural language input interfaces have been used individually, but the range of processing that the system can accept is narrow, and the user cannot immediately perform the desired operation. There was a problem that it could not be executed. This is for the output results.

One of the reasons is that the user cannot directly operate it. Based on the knowledge (data) currently available, the user
For example, you want to know more details.

It is thought that there are many cases where you want to directly manipulate what you think, such as when you want to know something related to it.

Conventional 5 It has been extremely difficult to construct an interface that satisfies this requirement.

Furthermore, constructing a user interface using multimedia required a great deal of time and effort.

The present invention aims to solve the above problems and provides a means for easily and uniformly constructing a visual and direct graphic interface by managing the display area of objects, a natural language interface by using a dictionary, etc. The purpose is to

[Failure to solve the problem]

FIG. 1 is a diagram explaining the principle of the present invention.

In FIG. 1, 10 is a display, 11 is a processing unit including a CPU and memory, 12 is an object having attribute values and methods, 13 is an analysis dictionary used for natural language analysis, and 14 is an object display interface. 15 is an object display unit that displays a specified object; 16 is an access table registration unit that registers a display object in an access table; and 17 is an input system that provides a unified human interface. Processing means, 1B is an object extraction unit that extracts a specified object, 19 is a command generation/execution unit that generates and executes a command by natural language input, 20 is a menu processing unit that selects and executes a method from a menu, and 21 is a An access table in which objects and their occupied area information are registered; 22 is an input control unit that performs input control; and 23 is an input 'AH' such as a keyboard or pointing device.

The present invention is based on a data processing system that handles an object-oriented model, in which each object 12 has display information such as a figure, text, or image, and occupied area information related to the display area with various predetermined attribute names. It will be done like this. In other words, if the object 12 is created to have predetermined display information, occupied area information, etc.

Display system processing means 14, input system, and processing means 1 to be described later
7 interface can be used.

The analysis dictionary 13 contains natural language words and object/
Correspondence information such as attributes, conditions, methods, etc. is registered in advance in a predetermined format.

By registering in the analysis dictionary 13 + fl information, the natural language interface by the input system processing means 17 can be used.

In response to an instruction to display an object 12, the display system processing means 14 refers to a predetermined attribute, and causes the object display section 15 to display the contents regarding the object and its component objects on the display 1.
Display at 0. Also.

The access table registration unit 16 performs a process of registering the identification information of the currently displayed object 12 and its occupied area information in the access table 21.

The access table 21 is a storage area for internal work provided corresponding to the window W to be displayed on the display IO, and stores information about each object displayed in the window W and its occupied area. It is something to do.

The input system processing means 17 receives the input from the three-manpower device 23 from the input control section 2.
2, when the coordinates on the object display screen are input, the object extracting section 18 searches the corresponding access table 21 and performs a process of extracting the object corresponding to the input coordinates as an operation target. In addition, analysis is performed with reference to the analysis dictionary I3 for the natural language human ability.

Generate commands using object manipulation language.

It has a command generation/execution unit 19 that executes the command, and a menu processing unit 20 that performs menu display processing for the method possessed by the extracted object and executes the method/7 code selected by the menu.

(Operation) When the figures, image data, etc. that are the targets of each fi! operation and the window are defined as objects according to a predetermined definition format, they can be displayed by the 9 display system processing means 14.Display system processing means 14 Since the input system processing means 17 automatically registers and manages information regarding all objects including the part objects to be displayed in the access table 21,
By referring to 1, all objects displayed in 1 can be recognized by inputting from a pointing device such as a mouse.

Furthermore, if the analysis dictionary 13 is created according to a predetermined format, the input system processing means 17 can provide a processing PI function that automatically converts the input natural language into an object manipulation language according to the registered information. Therefore, natural language interfaces such as Japanese can be easily created. Also, the method of the object selected from the displayed objects can be automatically selected from the menu, so the user can...

It becomes possible to instruct the processing device 11 to operate the currently displayed object using one of various styles of interfaces.

〔Example〕

No. 21E] is the access control used in one embodiment of the present invention.
An explanatory diagram of a table, FIG. 3 is an example of a display operation according to an embodiment of the present invention, FIG. 4 is an example of an analysis dictionary according to an embodiment of the present invention, and FIG. 5 is an example of an embodiment of the present invention. An example of command generation by example. FIG. 6 shows a processing explanatory diagram of an embodiment of the present invention.

In processing systems that handle object-oriented models,
As is well known, objects to be manipulated have attribute values and operation (method) information, and can be used to inspect/change attribute values, execute methods, send/receive messages, etc.
This can now be done by

In the present invention, in order to support display and various operations based on this object, the following information is provided in a common format (5) as attributes that the object 1- should have.
attribute name).

[The emotion that objects have 11’l ・Display information: Figure = O line (surface) type and coordinates, line (surface) attribute 2 colors, etc. information Text → string, font, starting position, direction.

Information such as color, background color, image pixel value of each pixel, matrix, color,
Information such as data length - Parent figure (part figure) Hierarchical relationship information such as configuring two parts - Occupied area (display area): Information on occupied area in the parent part's coordinate system in a part relationship - Relative position Parent in a two-part relationship Relative position information in the coordinate system of the part - Relative magnification In the relationship between two parts, scale ratio information with the coordinate system of the parent part - Position on the window, - Magnification on the window, -
Occupied area on the window...

Note that the values of these window-related information are set when the occupied area is calculated in a single display system. The occupied 3■ area is managed as a rectangle.

[Window Information] Objects related to windows have the following secondary attribute information.

・Position Position and size on the second screen Size on the second screen ・Access ・Table: A table of the identifiers of objects displayed in the window and their occupied areas ・Top flag: Overlapping of windows 5.Flag to identify whether it is on the front or the back ・The position of the view, ・The size of the view (Use this to move or reduce/enlarge the entire shape on the window) ・Background color... ....

The display system processing means 14 shown in FIG. 1 displays the specified object in the specified window based on the information of two or more objects, and also displays the access information corresponding to the window in which the object is displayed.
Object information is registered in the table 21.

Figure 2 (a) shows the map display window W.
Figure 2 (b) shows the corresponding access table 2.
Example 1 is shown.

The access table 21 stores, for all objects displayed in the map display window W, object identifiers and occupied area information by a rectangle defined by the upper left coordinates and lower right coordinates. Therefore, for example, when a point in the map display window W is designated by the mouse cursor 30, the access table 21 determines which object was selected (t?). 2 and can send a message to that object (for example, house B).

As an example of display operations, object traversal and browsing will be described with reference to FIG.

Traversing means tracing and searching for an object, and browsing means displaying detailed contents of an object.

For example, in the result window W shown in FIG. 3(a), if you specify "dwelling 2" with a mouse or the like, the object containing the input coordinates will be searched by referring to the access table 21 of that window. is,
The object "Dwelling 2" is selected. This results in
You will be able to send messages to "Residence 2".

When you command to display all five attributes using the method prepared for "Dwelling 2", as shown in Figure 3 (b),
All attribute information of "Residence 2" is displayed in the result window W. A message can be sent to an object therein (for example, apartment A) in the same manner as in 1, and as shown in FIG. 3 (c), it is also possible to browse "apartment A." Furthermore, by sending a message "Please display 1" to the image object "Photo P", the exterior photo of apartment A prepared in advance is displayed as shown in Figure 3 (d). You can also display
This can be done using similar operations.

How to send a message to an object selected with a pointing device.

For example, sending a command automatically generated by inputting Japanese, such as "Find the nearest supermarket."
Transmission can be selected by menu selection based on pre-prepared methods such as attribute inspection, deletion, movement, etc.

The analysis dictionary 13 used by the input system processing means 17 shown in FIG. 1 is configured as shown in FIG. 4, for example.

In this example, the analysis dictionary 13 includes arbitrary Japanese words for data handled by various objects, and type information such as imperative words and auxiliary words indicating the type of each word.
Information related to the object is registered in advance. Information related to objects includes the identifier of the object when the word means an object, the name of the attribute when the word means one of its attributes, the search condition, etc. , condition information corresponding to the operating language, and, if it corresponds to a method, the identification name of the method. As long as the user follows the prescribed format,
Any word can be registered in the analysis dictionary 13.

Next, an example of command generation using this analysis dictionary 13 will be explained with reference to FIG.

As shown in Figure 5(a), 2, for example, the floor plan is 3DK,
When a Japanese word such as "" is input, the word search in the analysis dictionary 13 shown in FIG. Depending on whether this is the case, the method section generation module or the condition section generation module is called.Here, the 1 condition section generation module is called, and from the word "3DK".

r ((madori) equal “3DK”)
A condition part of J is generated.

In addition, as shown in Figure 5(b), when Japanese such as "Find the price and floor plan of the property" is input, word segmentation is performed in the same way, and since it is an imperative sentence, the method Part-generated module is called. Then, “property” corresponds to the object juukyo, “price”, “
The analysis dictionary 13 makes it clear that "floor plan" corresponds to the attributes kakaku and madori, respectively, and "ask" corresponds to the method get.

When the analysis of the conditional part and the method part is completed, the nine conditional parts and the method part are combined according to the pattern according to the predetermined object manipulation language as shown in FIG. 5(c), and the result is as shown in FIG. Automatically generate object commands. Then run that command. Therefore, the user can construct a 5-Japanese input interface simply by registering words necessary for the application in the analysis dictionary 13.

FIG. 6 shows an example of processing related to an interface according to an embodiment of the present invention.

Display system processing is performed as shown in FIG. 6(a).

■ to ■ in the following description correspond to processes ■ to ■ shown in FIG. 6(a).

(2) In response to a display instruction specifying a window and an object, the area occupied by the two display objects and their component objects, including the view of one display window, is calculated from the window information and object information.

In this case, the relative position defined as an attribute of the object.

Information such as magnification and occupied area is used to further determine the position, magnification, and occupied area on the window.

■ Display the contents of an object based on the object's display information and its position and magnification information in the window. This display content also applies to 9 shapes.

It can be text or an image.

■ Rectangle information (4 coordinates) about the display area occupied by the displayed object and the object identifier.
Register in the access table.

Note that the registration order is that the parent figure comes first and the part figure comes later.

■ The input system process of repeating display and registration by process ■ and process ■ for all component figures and terminating the process when the display and registration process is completed is shown in Figure 6 (b). It is done like this. ■～ in the following explanation
0 corresponds to processes ① to ① shown in FIG. 6(b).

■ The system notifies you of input coordinates and input capotan by mouse input.

■ Determine the window from the input coordinates based on the occupied area information of the window information. The occupied area is.

It is given by four numerical values: the upper left x, y coordinates and the lower right x, y coordinates of a rectangle, and when a quintic equation holds true for the input coordinates X, Y, that window becomes a candidate.

Upper left X≦Manual power

(2) Next, an object whose occupied area includes the input coordinates is determined from the access table 21 of the determined window. As for comparison expressions.

It is similar to the formula used in determining the window.

In searching the access table 21 in this process,
Scan in reverse order of input.

In display processing, as described above, the parent figure is displayed first, so the nine child part figures are displayed later. From this, when the access table 21 is reverse scanned, even among objects occupying the same coordinates,
Child part shapes can be searched first (object extraction).

Furthermore, if the shape is not the object of operation.

For example, by right-clicking the mouse, the search for tertiary object candidates is continued (object selection).

In this embodiment, a left click of the mouse is used to access an object, a middle click is used to terminate, and a right click is used to select the next candidate.

If there are no candidates, go back to the beginning.

■ When the object to be operated is selected, the secondary Japanese processing or menu selection operation selection is performed. This is selected by turning on/off a specified variable that can be specified at any time.

■ In the case of Japanese processing 2. Input Japanese characters using well-known kana-kanji conversion.

(1) Using the analysis dictionary 13, the input sentence is divided into words by, for example, the longest match method, and analyzed.

■ Find the word to be executed from the dictionary information.

According to the analysis result, auxiliary information of each -word is adopted,
By combining them, a command in the Object 1 Kuwasaku language is generated. Note that if the user desires more advanced analysis, he or she may replace this part with another generation routine.

At this time, it is also possible to use a general-purpose program.

■ Execute the generated command, and when processing is complete, transfer control to the mouse input system.

■ If menu selection is specified by the switch of the specified variable, display the menu of methods possessed by the object to be manipulated.

[Phase] Select one item from the displayed menu of methods using mouse input or the like.

9. Execute the method of the selected menu item.

Once execution is complete, control is returned to the mouse input system.

In this embodiment, a visual interface is mainly used, but since two operation targets are managed in a unified manner as objects, it is easy to extend the system to handle media such as audio. Note that the object to be operated and the dictionary for analysis may be freely created by the user according to predetermined rules.

A generation tool that supports the generation of this object.

By also providing a dictionary creation tool.

Furthermore, it is possible to construct an application system more easily.

〔Effect of the invention〕

As explained above, according to the present invention, by simply defining knowledge (data) of the target field and information such as figures, windows, dictionaries, etc. necessary for the interface using an object-oriented model, it is possible to visually/directly It becomes possible to create an easy-to-use user interface that allows selection of various displays and operations such as object browsing (traversal).

Because it is based on an object-oriented model, it is machine independent and highly portable. Various media can also be managed uniformly as objects.

[Brief explanation of the drawing]

FIG. 1 is a diagram explaining the principle of the present invention. FIG. 2 is an explanatory diagram of an access table used in an embodiment of the present invention. FIG. 3 is an example of display operation according to an embodiment of the present invention. FIG. 4 is an example of an analysis dictionary according to an embodiment of the present invention. FIG. 5 is an example of command generation according to an embodiment of the present invention. FIG. 6 shows a processing explanatory diagram of an embodiment of the present invention. In the figure, 10 is a display, and 11 is a processing device. 12 is an object, 13 is an analysis dictionary, 14 is a display system processing means, 15 is an object display section, 16 is an access table registration section, 17 is an input system processing means, 18 is an object extraction section, and 19 is a command generation/execution section. 20 is a menu processing section, 21 is an access table, 22 is a human control section, and 23 is an input device.

Claims (1)

[Claims] In an object-oriented data processing system that manages an operation target as an object, in response to a display instruction for an object that has at least display information and occupied area information according to a predetermined definition format, contents regarding that object and its component objects are displayed. Display system processing means (14) registers display objects and their occupied area information in an access table (21) provided for each window, and a display system processing means (14) that registers display objects and their occupied area information in an access table (21) provided for each window; 21) means for extracting an object to be manipulated, means for performing menu processing for a method possessed by the object to be manipulated, and means for generating and executing commands in response to natural language input according to a dictionary created in a predetermined definition format. an input system processing means (1) having at least
7) A user interface creation tool comprising: