JPH0410030A - Program production method - Google Patents

Abstract

PURPOSE:To enable a user to intuitively grasp the executing state of a program by changing the display of an icon corresponding to the program according to the executing stage of the program and in relation to the icons showing other processing subjects. CONSTITUTION:The display of an icon 3102 corresponding to a program under production is changed in relation to the icons 3104 - 3106 corresponding to other processing subjects at a point near the time point when the program under production requests the processing that is already assigned to another subject. For instance, the icon 3102 is moved to the places of the icons 3104 - 3106. At the same time, the display is changed at the time point when the produced program is carried out. Furthermore an object projected on a two-dimensional plane within a three-dimensional space is used as an icon for input of a computer. Thus a user can grasp the executing state of the program and can perform the input of the computer with use of a screen having a very near relation.

Description

Detailed Description of the Invention 3. Detailed Description of the Invention [Field of Industrial Application] The present invention relates to the usage of a computer, a program execution method, and a program creation method.

[Prior Art] In recent years, a computational model called object-oriented has become widely used. A calculation model is the basic way of thinking and understanding when performing calculations, and when realizing this model with a computer etc., it is necessary to understand the data structure, data transfer, data conversion procedure, when,
It stipulates the timing at which it is to be carried out.

The terms used in the object-oriented computational model are explained below.

(a) Object This is a set of data structures and procedures. It has a slot (b) as a data structure and a method (b) as a procedure.

(b) Name of the data structure of the slot object. For example, an integer object has an integer value slot, and a document object has a document slot.

(c) Slot value The actual data held in the object's slots.

(d) Procedures defined in method objects. By only allowing operations on slots, program modularity is guaranteed.

(e) Object Instance Objects are treated conceptually, and objects that actually have slot values are called object instances, which are not actually placed on a storage device.

(f) Command for executing message method. It consists of arguments and method names, but by limiting arguments to object instances, it guarantees program modularity.

An object-oriented computational model is a computational model that allows computation only by exchanging messages between object instances. The result of the calculation appears as a change in the slot value of the object instance. For example, an integer object has an integer value slot as a slot,
It also has methods such as addition methods. To design an integer object instance A with a concrete value of 3, make its slot value 3. If you want to add 4 to this integer object instance A, prepare another integer object instance B and set the value of the integer value slot of 20B to 4.

Sends the message "B addition" to A. As a result, the value of the integer value slot of A becomes 7, which means that the desired result has been obtained. However, since it is cumbersome to generate object instances for all simple state quantities such as integers, simple state quantities are called atoms, and when they are used as arguments, their values are Allows to be given as is to other object instants. For example, in the above example, for A, “4
Sends an informal message called addition J to A.

By using this computational model, data and procedures are stored in the same place, making it possible to create programs that are highly modular and easy to understand.

For example, an icon representing an object instance is displayed on a display device, and an operator can use a mouse or the like to send a message to convert the state quantity of this object instance, or to move or transform this icon. This is widely practiced. For example, by pointing a certain icon with the mouse, you specify the corresponding object instance, interpret the subsequent input "(move amount) move" as a message for this object instance, and execute it. It moves the . As a result of this movement, the value of the slot representing its position within the object instance changes.

In addition to receiving and executing operator input as messages, programming is also performed using icons. That is, an object instance is displayed as an icon on a display, and a program is generated by operating the object instance from the user.

This is the Programming Pi Example (P r
o gr ammxng by Example
e:PBE), which displays object instances such as files and application programs, which are various computational resources, as icons on a display device, and allows an operator to manipulate these icons using a mouse or other device. , the system interprets this operation, performs this operation on the object instance pointed to by this operation, and simultaneously generates a command representing the operation. In this way, a desired program can be obtained by repeating operations, executions, and command generation.

An object instance has a specific value (example), but the system interprets operations on this instance and generates a program for a more general object. The generated program can be applied to an example different from the example operated by the user when creating the program.

This is because, for an object instance, the operational interpretation for this example is converted into a form that can be applied to everything belonging to that object. Furthermore, since programming is performed using specific examples, the programmer does not need to generalize the problem as in normal programming, which reduces programming effort.

As an example of PBE, a person called an agent is defined as an object instance corresponding to a program that manages program startup, and this is displayed as an icon on a display device, and instructions are sent to it to execute the program. A system with an interface for requesting is published in Nikkei Computer "User Interface in the Era of Software Multimedia", Pp, 165-178 (published on July 28, 1989).
(hereinafter, this technique will be referred to as the first known technique). This system displays an icon consisting of a two-dimensional figure with a human face representing an object instance of a management program called an agent on a display device, and an icon consisting of two-dimensional figures each representing an object instance representing a file, a cabinet, etc. Icons are displayed, and the operator gives instructions to the displayed agent icon and other displayed icons using a mouse or keyboard. The agent interprets input from the user and selects and executes the appropriate program.

The function of the PBE in this system is to store the result of interpreting the input from the user by attaching it to the requested agent icon, and to generate an input string as a program.

Information Processing Society of Japan's 30th Programming Symposium Reports, pp. 2012, pp. 11-11, as a technology that does not use the concept of objects or methods, but uses icons to facilitate program creation.

9-17 (January 1989), the program generation system described by the inventors (hereinafter referred to as the second known technology)
There is. Here, examples of data such as numerical data and array data are displayed as icons on the display device, and the corresponding icon is selected on the screen to specify the processing target of the input command. There is. Additionally, the commands input when creating the program are executed on the example data, and the results are notified to the operator. Also, depending on the results, the displayed icon may be divided into multiple areas. By displaying the execution status of the program in this manner, the current execution status of the program being created can be accurately and easily notified to the operator.

Furthermore, in this system, if the execution of the program being created fails, it is possible to create a program consisting of another command sequence.

In addition, the IEEE Workshop on Visual Language (IEEE Worksho
p on Visual Language
e), pp. 80-85 1989 (hereinafter referred to as the third prior art) includes the following technology.

A technique is displayed in which a disk forming the Tower of Hanoi is displayed as an example of a processing target, and the disk is used like an icon to create a program in a manner similar to the second known technique. Here, when creating a program, a command to move the disk is input as part of the command, and this command is executed. What's more, this movement is shown as a video. This makes it easier to understand the program execution status.

[Problems to be Solved by the Invention] In the second or third known technique, the operator is informed of the execution stage of the program by changing the display of the icon corresponding to the processing target depending on the execution stage. but,
The second and third known techniques do not have the concept of objects, methods, or messages, as is the case with the first known technique. In other words, all you have to do is create a program consisting of commands that directly specify the processing target. On the other hand, in the first known technique, as a method (program) to be created for a certain object, it is possible to create a program that executes processing including processing by a method assigned to another object. In that sense, it can be said to be an extremely versatile or advanced program creation method. but.

In the case of this first known technique, the execution state of the program being created cannot be known.

Therefore, an object of the present invention is to provide a program creation method that allows an operator to know the execution state of a program that executes processes including processes that have already been assigned to various processing targets.

Another object of the present invention is to provide a method of executing a program having the same characteristics. Furthermore, another object of the present invention is to provide a method of displaying icons that facilitates computer input.

[Means for solving the problem] In order to solve the above problem, a program being created corresponds to the program being created near the time when the program being created requests processing that has already been assigned to another processing target. The display of the icon 9 is changed in relation to the icon corresponding to the other processing target.

For example, move the former icon to the latter icon. Further, in the execution stage of the created program, the above-mentioned display is changed at the above-mentioned point. Furthermore, as an icon for computer input, a projection of an object in a three-dimensional space onto a two-dimensional plane is used.

[Effect] Depending on the execution stage of the program being created or already created,
Since the display of the icon corresponding to the program is changed in relation to the icon for the processing target to which the requested processing is assigned, the user can know the execution status of the program.

Additionally, by using three-dimensional icons, computer input can be performed using a very close-looking screen.

[Embodiment] A. Overview FIG. 1 is a diagram showing the configuration of a program generation system according to an embodiment of the present invention. As shown in this figure, the system includes a display device 101, a mouse 109, and a keyboard 1.
10 and a voice input device 111, and a main storage device 1 that stores a program 10 for creating a program.
12, an auxiliary storage device 113, a CPU 114, a network device 115 for connecting to a large general-purpose computer 116 and a telephone line 117, and an audio output device 118.

The display screen of display device 101 consists of screen areas 102 to 107 as shown in the figure.

The object display screen 102 shows various equipment in the office space, such as a telephone 3104, a desk 3105, a terminal 3106, and a person 3 representing a secretary standing there.
1o2 at a certain point. A three-dimensional figure as viewed from a virtual TV camera (not shown) is displayed in advance. In this embodiment, an object in the object-oriented calculation model is defined for each of these figures. These figures are used as icons of the respective objects. A desired program is created as a method for the person 3102 using these icons. In order to make the operator aware of the execution status of the created program when it is subsequently executed, in this embodiment, the person 3102 is made to perform an action corresponding to the execution status of the program when the program is created. Enter a command that instructs this operation as part of the program to be created.

As will be described later, programming is performed while moving this person 3102. The motion control screen 103 includes a slider 217 for instructing this movement.
~222 is displayed.

The view controller screen 104 includes sliders 230 to 2 for controlling the position or posture of the virtual VT camera in order to change the view of the office space.
35 is displayed. Method programming screen 105
Displays the commands entered to configure the program to be created. Program control screen 1
06 displays a menu and the like for controlling execution or storage of commands input by the operator.

On the terminal screen 107, there is an input device! 109 to 111, control information that informs the operator of the status of the terminal consisting of the display device 101 is displayed. The object modeling screen 108 displays information regarding the object represented by the icon selected by the operator from among the various icons displayed on the object display screen. 20
1 indicates a mouse cursor shown on the display screen.

In Figure 2, 119, 120, 126 to 136.1
Reference numerals 40 to 413 indicate various modules of the program creation program 10 prepared in advance. There is.

The following three databases are stored in advance in the main memory W112 for this program creation program 10.

(1) Object instance library 124 that stores information about each object, that is, object instances. Each object instance consists of structural data, object slots, and object methods of each object.

Examples thereof are shown in Figures 3A-3C.

(2) Three-dimensional shape database 123 which is a collection of graphic data of three-dimensional icons representing each object. An example of the graphic data is shown in FIG.

(3) Candidate speech database 125 used for speech recognition from the speech input device 111 An example of this data is shown in FIG.

These data are originally stored in the auxiliary memory 113, but when needed by the program creation program, they can be stored in the main memory! placed in 112.

An overview of the processing of the program creation program 10 will be described below.

(1) User input information from three input devices, the mouse 109, the keyboard 110, and the voice input device 111, is interpreted by processes 119, 126, 127, and 140 to 143 to create commands and stored in the command buffer 120. , which is added to the command queue 121 at the top of the command queue stack 122. The above process is the second
Processes 119, 126, 127, 140-143 in the figure are performed. A person icon 3102 is used as a command.
A command for a desired operation is also input. At this time, various sliders on the motion control screen 103 are operated.

(2) In object method execution processing 135, a command is taken out from command queue 121 and executed with reference to three-dimensional shape database 123 and object instance library 124. In other words, if this extracted command has a defined method for the object to which the specified object instance belongs, an appropriate program library specified by the method is executed. For example, when the command is a command to move the person icon 3102, the display of the three-dimensional icon corresponding to the object is appropriately changed into a moving image. This allows the user to know which command was executed when creating the program. When displaying this video, the execution process 135 updates the three-dimensional shape database regarding the object as appropriate according to the program library, and the object display process 125 changes the display on the object display screen 102 in immediate response to the updated result. . In this way, by executing the command, a 3D image is displayed as a moving image, and
The audio output device 118 also outputs audio to notify the operator of the execution status as the command is executed.

(3) If the method requested by the command is not defined among the methods of the object specified by the command, a new method is generated using object method execution processing 135.

(4) If any problem occurs as a result of executing a command, a command failure is reported.

In this case, the method specified by the executed command is reprogrammed. This is also performed using the object method execution process 135.

In this way, the object method execution processing 135 performs various functions depending on the situation, and the command queue stack 122 functions to control this.

(5) The position and display state of the three-dimensional shape icon corresponding to the object instance can be changed at any time. This is motion control screen 10
This is done by moving the valuators (vertical thick line parts) of the sliders 217 to 222 on the top 3 left and right with the mouse. The results of changes are not stored as instructions in the program being created during programming. When the "memory" button 216 on the program control screen 106 is selected with the mouse, a command sequence is generated that reproduces the movement or position change leading up to the change, and is stored in the program being created.

(6) Similar to (5), the state in which icons having a three-dimensional shape are projected onto the object display screen 102 can be changed at any time. Similarly to (5), by selecting the memory button 216 with the mouse, a command to reproduce that state is generated and stored in the program.

As described above, one feature of this embodiment is that, during program creation, a command that causes the person icon 3102 assigned to this program to make a desired movement is input as part of the commands belonging to the program to be created. Furthermore, when creating a program, this is executed and the operator is informed of the execution status of the input command in terms of programming. Furthermore, in this embodiment, three-dimensional figures are used as they are as icons, including this person icon. Other features of this embodiment will be explained in the detailed description below.

B. System Components Before describing the details of the processing of the system of this embodiment, the details of each component will be described below.

B.1 Database 1, 3D shape database 123 (Fig. 4) In this embodiment, three-dimensional figures representing people, telephones, desks, etc. placed in an office space are acquired from a camera placed in the space. The graphic as viewed is displayed on the display screen, and the displayed graphic is used as an icon to represent the corresponding object.

The three-dimensional shape database 123 stores data on such three-dimensional figures. As illustrated in FIG. 4, the data for each three-dimensional figure includes the types and positions of the plurality of polyhedra constituting each three-dimensional figure, the color of each polyhedron,
It consists of attributes such as gloss and a transformation matrix. FIG. 4 shows telephone object instance 2 shown in FIG. 3B.
3D graphic data 27 of a receiver object instance 2754 which is a lower element of 750
Part of 99 is shown.

Here, an octahedron, its vertex coordinates, color, gloss, position, and direction are shown.

2. Object instance library 124 (Figures 3A to 30) The object instance library 124 stores information on various objects (object instances).
, hold object structure data, object slots, and object methods, for example. 3A to 3C show specific examples of object instances corresponding to a person, a telephone, and a camera, respectively. Here, explanation will be given taking the person object 2701 in FIG. 3A registered in the object instance library 124 as an example.

A slot 2711 is the name of an attribute of the person object and has a slot value. Slot 2711 specifies the name of each attribute and the data type for that attribute. In the example of FIG. 3A, the slot 2711 of the person object instance 27o1 has three attribute names: position 2720, direction 2721, and name 2722, and the data types for each are vector, vector, vector, etc.
Indicates that string type data is stored.

An object method is an object-specific procedure, and here it is a set of its name, multiple command strings corresponding to the name, and argument objects. Method calls at runtime are performed in the order of definition.

In the example of Figure 3A, the call method 2712, go
to method 2713, tel take method, 2714, connect (1) method 271
5. CONNECT (2) Method 27
16 are defined. Here, the C0nnec method has two definitions, and they are ordered to represent the order of definition.

In this embodiment, the object has elements configured in a tree structure. Each element is also an object. The same applies to object instances.

As can be seen from the object structure data 27o2 in FIG. 3A, the person object instance 2703 is the body object instance 270.
4 as a lower element. A body object instance 2704 includes a right upper arm object instance 2708 and a left upper arm object instance 2708.
m) It has an object instance 2705, etc. as lower elements. Among these, ruarm object instance 2708 has rlarm object instance 27o9 as a lower element, and rlarm object instance 2709 has rhand object instance 2710 as a lower element. Specific examples of object instances of elements are omitted from illustration and detailed description for the sake of brevity.

In this embodiment, each element has a three-dimensional shape.

The 3D shape is stored in the 3D shape database 123 (Figure 2)
is stored in. Furthermore, the object instance (not shown) of each element holds a movable attribute and a rotatable attribute as attributes of that element. In the example of FIG. 3A, an element with a movable attribute is marked with a (+), and an element with a rotatable attribute is marked with a (book). This also applies to other figures, for example, figures 3B and 3C.

If the movable or rotatable attributes of an element are not movable or rotatable, they cannot be moved or rotated, respectively. In the example of FIG. 3A, all of the subelements of person object instance 2703 do not have a movable attribute.

This creates a natural state in which the human joints can only rotate and cannot move.

An instance for a special object is camera object instance 2780 (Figure 3C). Camera object instance 2780 has the camera position as its slot 2790 [2791
, a lateral direction 2792 indicating the horizontal direction of the field of view of the camera;
3, upward direction 2793, indicating the vertical direction of the camera's field of view.
Store vector type data in each of the two attributes. Also, this object instance has method 2795
has as a built-in method. This method converts the data of various three-dimensional figures in the three-dimensional shape database 123 into coordinate system data on the object display screen 102 (Fig. 1), performs shading and rendering, and displays the obtained figures as objects. Display on screen 1.02.

Note that the object display processing 136 (Fig. 2) executes this method when the three-dimensional shape database 123 or object instance library 124 is updated, or when there is another screen display request. There is.

The object instance library 124 is modified when an object instance is created or deleted.

3. Candidate speech database 125 (FIG. 5) The candidate speech database 125 is a database that stores a plurality of candidate speech data 501 shown in FIG. 5, which is used when recognizing speech input from the speech input device 111. . Here, the candidate voice data 501 is composed of areas of voice data 502, data type 5o3, and character string 504.

The voice data 502 is the voice input stored as is, the data type 503 represents the type of candidate voice data, and the character string 504 is a character string output as a result of voice recognition. Here, the data type 503 represents any one of an object instance, an object method, a built-in instruction code, and a program control.

The candidate voice data registered from the beginning includes the following.

(1) Candidate voice data regarding the built-in command (FIG. 3), which will be described later. This data type is an embedded instruction code.

(2) Candidate audio data related to object instances that are originally defined as built-in object instances, such as camera object instances. These data types are object instances.

(3) Candidate voice data corresponding to the operator's voice input of “execute”. The data type of this candidate audio data is program control, and the character string is "eXecute".

(4) Candidate voice data corresponding to the operator's voice input of “memory”. The data type of this candidate audio data is program control, and the character string is rec a rd''.

8.2 Processing Below, processing by various programs within this system shown in FIG. 2 will be described.

1. Input to the input processing system is through keyboard 109 and mouse 1.
10, performed using the voice input device 111.

1.1 Event data (Figure 6) As described later, voice event generation processing 143 and mouse event generation processing 14 are performed in response to input to the system.
0 and keyboard event generation processing 141 generate event data.

The event data 401 is composed of an event type area 402 and an additional information area 403, as shown in FIG. Here, the event type 402 is one of motion control, view control, object selection, program control, command input, object modeling, or keyboard. Additional information 403 differs depending on the event type. When the event type is object selection, command input, or keyboard,
The event data is used to compose commands or parts thereof constituting the program to be created. Other types of event data are not directly related to command generation.

1.2 Voice Processing 1.2.1 Voice Input From the voice input device 111, the operator inputs a process by voice as if instructing the person icon 31o2 to perform a process that the program to be created should execute.

For example, enter "Pick up the phone." In this voice input processing, this input voice is recognized and one or more necessary event data are generated. Not only a voice instructing a process but also an input related to the execution of a created command, such as "execute", can be input from the voice input device 111. However, regardless of the type of voice, the expression must be registered in advance.

1.2.2 Speech recognition processing 142 (Fig. 7A) In the speech recognition processing 142, as shown in Fig. 6, the voice input device 11
1 and all the candidate voice data 5 in the candidate voice database 125.
01 (FIG. 5), and extract candidate audio data that is most similar to each audio portion. In step 601, audio event generation processing 143 (FIG. 2) is performed for each audio portion. Start (step 60)
3). If there are no similar candidate voices (step 602), nothing is output.

1.2.3 Voice event generation processing 143 (Figure 7B) As shown in Figure 7B, the data type area in the candidate voice data extracted by the voice recognition process 142 is examined (steps 701, 703, 705, 707). 702, 704, 706, 708) and activates the input event processing 126 (FIG. 2) (step 709).

The event data generated is as follows.

Note that when generating event data, the character string area 504 of the extracted candidate audio data 501 is used as is for the additional information 403.

For example, the data type of candidate voice data extracted for a voice portion representing a processing target, such as ``telephone'' in the input voice ``Pick up the phone'', is an object instance. In that case (step 701), event data whose event type is object selection is generated (step 702).

For example, the data type of candidate voice data extracted for a voice portion representing processing content, such as ``pick up'' in the input voice ``pick up the phone'', is an object method. In this case (step 703), event data whose event type is command input is generated (step 704).
).

The data type of candidate voice data extracted for a portion representing a built-in command (described later) in the input voice is a built-in command code. In this case (step 705), event data whose event type is command input is generated (
Step 706).

The above three event data are generated when an operator inputs by voice an object instance or an object method as a processing target or an argument of a command to be generated, or an instruction code of the command.

For example, the data type of the candidate voice data extracted for the input voice "memory" or "execution J" is program control. In this case (step 707) = event data whose event type is program control is generated (step 708).

The event data for the input voice "execute" instructs the execution of the command generated immediately before, but is not used to generate the command. "Memory" is used to generate commands.

1.3 Mouse input and mouse event generation processing 140 (
(FIG. 8A) Display screen 102 in the display device 101 in FIG.
1o4. Mouse cursor 2 on either 106 or 108
By placing 01, input using the screen becomes possible.

Mouse event generation processing is a general term for event generation processing by the mouse, and includes object selection event generation processing 304 and motion event generation processing 305 described below.
．． View event generation processing 306, program control event generation processing 307. It consists of object modeling event generation processing 308. These processes 304
~3.8 are activated when the mouse cursor 201 is located on the display screens 102, 103, 104, 106, and 108, respectively, as shown in FIG. 8A.

Note that object selection event data generation processing 304
The event data generated in 305, 306, and 308 is used to generate commands, but the event data generated in other processes 305, 306, and 308 is not used to generate commands. The event data obtained by processing 307 includes some data that is used for command generation and some that is not.

1.3.1 Object selection event processing 304 (
(FIG. 8B) This process is executed when the operator uses the mouse to input an object instance to be used as a processing target or argument of a command to be generated. That is, the operator places the mouse cursor 20 on the icon in the object display screen 102 that represents the object instance that the operator wishes to input.
1 and press the mouse button (not shown). In response to this mouse operation, object selection event processing 304 is activated. As shown in Figure 8B, first, 3
The object instance selected by this mouse operation is recognized by referring to the dimensional shape database 123 (step 801), and the event data containing the object selection as the event type and the character string representing the selected object instance as the additional information is stored. Generate (step 802). If the selected object instance is a component of another object instance, this additional information is a string consisting of the name of the superior object instance and the name of the selected object instance, separated by periods (,). Store. and input event processing 126 (Figure 2)
(step 803).

1.3.2 Motion event generation processing 305 (FIG. 8C) This processing is activated when the operator uses the motion control screen 103 to instruct movement (or rotation) of an icon within the object display screen 102. That is, as shown in FIG. 1, six valuators 217 to 222 are displayed on the motion control screen 103.

When any of these valuators is operated by the mouse 109, as shown in FIG. 8C, the event type is motion control, the additional information is the name of the operated valuator, and the scale on the support guide of the valuator slider. Event data storing the value (that is, the amount of movement or amount of rotation) is generated (step 901).

Then, input event processing 126 (FIG. 2) is activated (step 9o2). Note that before this variator operation, an object must be selected on the object display image 102.

1.3.3 View event generation processing 306 (8th D
Figure) In this process, the view control screen 10 is used for mouse input.
Motion event generation processing 30 except for generating event data in which view control is stored in the event type using six variators 230 to 235 in 4.
Perform the same process as in step 5 (Figure 8D, step 1001.1).
002).

1.3.4 Program control event generation process 307 (Figure 8E) When the mouse button (not shown) is pressed while the mouse cursor 201 is located above the "Run" button 215 (Figure 1) in the program control screen 106. If the 8th
As shown in Figure E, event data is generated in which the event type is program control and the additional information is "execute."

This is used to direct the execution of the most recently completed command. This is similar to the case of "execution" by human voice. Similarly, when the "Remember" button 216 is pressed, the event type is program control, and the additional information is "rec".
event data storing "ord" is generated (step 1101). In either case, the input event processing 126 (FIG. 2) is activated (step 1102).

The "memory" button 216 is pressed when an icon or camera (not shown) displayed on the object screen 102 is moved or rotated, and then a command to execute the movement or rotation is generated. Voice input "memory"
is also used in the same way.

1.3.5 Object modeling event generation processing 308 (Figure 8F) As shown in Figure 8F, object modeling screen 1
The mouse input at 08 generates event data in which object modeling is stored as the event type and input information is stored as additional information (step 1201). Then, the input event processing 126 (FIG. 2) is activated (1202). The event data shown here is used for creating or modifying an object instance, and is not directly related to creating a command.

1.4 Keyboard event generation process 309 (FIG. 9) Input from the keyboard 110 (FIG. 1) is sent to the keyboard event generation process 141 for each key operation.

As shown in FIG. 9, when the mouse cursor 201 is on the object modeling screen 108 (step 1
301) - Generate event data in which object modeling is stored as the event type and a key code is stored as additional information (step 1302).

Mouse cursor 201 is on object modeling screen 1
08 and the key code does not include a line feed code (1303), event data is generated in which the event type is the keyboard and the additional information is the key code or key code (step 1304). Key input without a line feed code input indicates that the input is information necessary to create a command, but is not the final input for command creation. Therefore, the additional information of this event data is used as part of the data of the command to be generated.

Mouse cursor 201 is on object modeling screen 1
08 and the key code does not include a line feed code at the end (step 1303), event data is generated in which the event type is a command input and the additional information is empty (step 1305).

The line feed code is input by the operator to indicate the last information necessary for command generation, that is, the input of the instruction code has been completed. Therefore, this event data indicates that the command should be completed with the previous input data.

Then, input event processing is started (step 130).
6).

1.5 Human Event Processing 126 The human event processing 126 includes a voice event generation process 143 and a mouse event generation process 140 as shown in FIG.
When activated by each event generation process of the keyboard event generation process 141, the event data 401 generated therein is added to the input event queue 119.

The incoming event queue 119 is implemented as a first-in, first-out queue, and the incoming event processing 126 stores event data starting at the end of the incoming event queue 119. On the other hand, an event dispatch process 127 for extracting event data, which will be described next, extracts event data stored at the beginning of the input event queue 119. This allows the input data to be processed in chronological order without being corrupted.

2. Interpretation and Processing of Input Events Each of the event data stored in the input event queue 19 as described above is interpreted and processed one by one by the event dispatch processing 127.

2.1 Command (Figure 10) The format of the command 1701 that constitutes the program to be created is as shown in Figure 17. That is, it consists of three parts, each consisting of a character, and each part is separated by a blank character.

(1) The first part 1702 is symmetrical in processing.

This corresponds to an instruction object in the object-oriented model. The name of the object instance to be processed is stored here.

This object instance stores either the name of the object instance of the object itself, the name of a component of that object instance, the name of a slot of that object instance, or a string indicating whether the object to be processed does not exist. be done.

If there is no target to be processed, it is limited to special built-in commands.

Here, the component name of an object instance is expressed by placing a period (,) between the object instance name and its component name, and the slot name of an object instance is expressed by placing a period (,) between the object instance name and the slot name. It is expressed by putting a sharp (#) next to it. In this way, a name is given uniquely within the system.

(2) The next part 1703 is an argument. Argument 1703 consists of an argument object with an arbitrary value.

Argument objects are separated by spaces.

The number of arguments varies depending on the method. The 6-argument object is either the object instance described above or an atom. Atoms can be integers, floating point numbers, or strings.

(3) The third part 1704 is an instruction code.

The instruction code 1704 is either a method name of an object instance or a built-in instruction code.

The command buffer 120 is a temporary storage location for commands 1701, and as described later, one or more event data or a portion of each event data is stored in a front-justified manner to complete one command.

However, the event data used for command creation is limited to event types such as object selection, command input, or keyboard.

2.2 Command Queue 121 (FIG. 11) As shown in FIG. 11, the command queue 121 holds a plurality of commands completed using the command buffer 120 as a first-in, first-out queue. When a command is added to the command queue 1801, it is added next to the last command added, and when a command is taken out from the command queue 1801, the first command added is taken out.

The command queue 1801 is created or deleted depending on the execution status of the program.

2.3 Command queue stack 122 (FIG. 12) As shown in FIG.
2 is a stack that stores a plurality of command queues 121, and only one is prepared in the system. Operations on the stack include push and pop; push adds a new command after the most recently pushed command queue, and pop takes out the most recently pushed command queue.

Also, at the bottom of the stack is an initial command queue 1903.
is placed.

2.4 Event dispatch processing 127 (Fig. 13) As shown in Fig. 13, the event dispatch processing 127 extracts event data one by one from the head of the input event queue 119 (Fig. 2) in step 1401.
), the processing of the event data is selected as follows depending on the event type. In addition, the event queue 119
If is empty, do nothing.

If the event type is keyboard (step 1402),
This event data is stored in key bond event generation processing 3.
As described for 09, although this is data for command creation, it is not the final data for creation. In other words, the key code of the additional information of this event data represents a processing measure or an argument. Therefore, the key code of this additional information is stored in the command buffer 120 (step 1403). The storage location is an empty area as far forward as possible within the command buffer 12o.

On the other hand, if the event type is object selection, command input motion control, view control 1, program control, object modeling (
Steps 141Q to 1415) Object selection processing 130, command input processing 131, motion control processing 128, view control processing 1291.
Program control processing 132. Activate the object modeling process 133 (step 1404-14)
09). Object selection processing 130, command input processing 1
At step 31, event data is stored in the command buffer 120 in order to create a command as described later.

Note that step 1403 or step 1410-
1415 is completed, the input event queue 119 is returned.
The process returns to the processing of the first event in FIG. 2.

2.5 Object selection process 130 (FIG. 14) When the event type of the event data interpreted in the event dispatch process 127 is object selection, this process 130 is activated.

As described above, such event data is generated when the operator uses the mouse to select an object instance to be processed or an argument of a command.

In the object selection process 130, as shown in FIG. 14, a blank character string is added following the additional information (this represents the processing target or argument) in the event data sent from the event dispatch process 127, and the command buffer 120 is (Step 200
1).

2.6 Command input process 131 (FIG. 15) When the event type of the event data interpreted by the event dispatch process 127 is a command input, this process 131 is activated. As mentioned above, this type of event data is generated by inputting the last information (i.e., instruction code) used for the command to be created at the same time or after inputting it, and is Input device [
111 or in response to an operator's input from the keyboard.

In the command input process 131, as shown in FIG. 15, the additional information (the end of which is the command code) of the event data sent from the event dispatch process 127 is front-aligned into the command buffer 120 to complete the command (step 2101).

Furthermore, this completed command is added to the end of the command queue at the top of the command queue stack 122 (step 2102), and the command buffer 120 is cleared (step 2103).

2.7 Motion control processing 128 (Fig. 16) This processing is activated when the type of event data interpreted by the event dispatch processing 127 is motion control.6 As already mentioned, this type of event data is The operator uses the motion control screen 10
Operate the valuator in 3 to display object surface screen 1.
It is activated when moving or rotating the icon in 02.

As described above, before this valueter operation, an object selection must be made to specify the icon to be moved or transformed. That is, as a result of this object selection, the name of the object instance to be motion controlled must already be stored in the processing target section in the command buffer 312o.

In this motion control processing 128, the icon for the object instance specified by this processing target section is stored in the additional information of the event data given from the event dispatch processing 127, and is moved or rotated according to the valuator name and the amount of movement or rotation angle. do. Movement or rotation is performed using the 3D shape database 123
This is achieved by modifying the graphic data within.

Note that if no processing target is stored in the command buffer 120, the following processing is not performed.

More specifically, as shown in FIG. 16, if this valuator name is X direction movement 217 (step 150
2) Check the object instance attribute of the processing target from the object instance library 124, confirm that it is "movable" (step 1505), and create an icon corresponding to the object instance indicated by the processing target according to the value of the valuator. It is moved in the X direction (step 1506).

Even if the attribute to be processed is not movable in step 15o5, if the target to be processed is a component of some object, the attribute of the element above the component is checked from the object instance library 124, and if it is movable, The element above it is determined to be movable and is moved. If there is nothing that can be moved even after tracing the upper-level components of the object, no processing is performed.

If the valuator is moving in the X direction 218 or moving in two directions 219, the same process is performed (steps 1503-1506
).

If the valuator name is Rotation around the X axis 220, check the attribute to be processed (step 1507),
After confirming that (step 1510), the processing target is rotated around the X axis according to the value of the valuator (step 1511).

Note that even if the attribute of the processing target is not "rotatable" in the determination in step 1510, if the referent is a component of an object, the attribute of the element above the component is checked and the attribute of the element that is "rotatable" is determined. , the referent is ″
It is determined that rotation is possible '', and in step 1511, the pointed object is rotated by rotating this higher-order element.

If there is no "rotatable" item even after tracing the upper-level components of the object, no processing is performed. The above rotation processing is performed by referring to the contents of the object library 124. Bali 221 rotates around the X axis 221. Rotation around the z-axis 222 is similarly processed (steps 1508-1511).

2.8 View control processing 129 (Fig. 17) The view control processing 129, as shown in Fig. 17, determines the position of the camera (not shown), the type of valuator operated in the horizontal direction and vertical direction of its field of view, Control according to the slider scale of the valuator (step 1601-
1604.1650°1651). The six valuators 230-235 (FIG. 1) represent a forward command, a rise command, a horizontal movement command, a head-up command, a swing command, and a rotation command, respectively.

For example, when the forward valuator 230 is operated (step 1601), the position slot value 2791 of the object instance camera 2780 (FIG. 3C) is moved in the direction of the direction slot value 2792 of the additional information value of the event data. Change according to the value of the slider scale (
Step 1602).

A desired view can be selected on the object display screen 102 by moving the object instance camera 2780 one rotation.

This movement or rotation is performed in the object library 124.
．． As in the case of motion control processing 128, this is performed with reference to the three-dimensional shape database 123, and the three-dimensional graphic data for the object is updated after movement or rotation.

2.9 Program control processing 132 (FIG. 18) This processing is activated when the data type in the event data interpreted in the event dispatch processing 127 is program control. This type of event is generated when "memory" or "execution" is instructed by voice input, mouse input, or manual keystroke, as described above.

This program control processing 132, as shown in FIG.
If "c o r d" (step 2201), start the state storage process 134 (FIG. 2) (step 2204).
, if the additional information is “execute” (step 2
202) Activate the object method execution process 135 (FIG. 2) (step 2203).

2.10 Object modeling process 133 (19th
Figure) This process is activated when the borrowed data type in the event data interpreted in the event dispatch process is object modeling. This type of event data is generated using mouse input when an object is newly defined or modified, as described above.

Object modeling process 133 is an independent application program. Here, the first
As shown in FIG. 9, a set of polyhedra representing each object and their respective attributes are newly defined (step 260
1). Next, this object is constructed hierarchically from a plurality of elements as illustrated in FIGS. 3A to 36, and a new object is defined for each element. At this time, whether the component is a movable attribute or a rotatable attribute is specified (step 2602). Next, the object name, object slot name, and data type to be stored in the object slot are specified (step 2603). The information about the figure representing the object and the information about the object itself thus created are stored in the three-dimensional shape database 123 and object library 124, respectively (step 2604).

At any time, it is also possible to edit objects that have already been defined in the object modeling process 133.

For example, as a subordinate element of the right hand 2710, which is a subordinate element of the person object 2703 in FIG. 3A, a telephone 2
If a hierarchy is created to register the handset 2805, which is a lower element of 8o3, the handset will operate according to the movement of the human hand, and as a result, it can be realized that a person has picked up the handset. The built-in instruction 1ink, which will be described later, is implemented in this way.

The name, shape, and attributes of the object defined in the object modeling process are displayed on the object modeling screen 10.
8 (FIG. 1) in areas 236°, 237.238. When modifying an object that has already been created,
If there is a method defined for the object, the name of the method is displayed in area 239.

3. State storage processing 134 (FIG. 20) As already mentioned, this processing is performed when the operator instructs storage of a created command.

It is activated by program control processing 132.

In this state storage process 134, first the command buffer 1
Check whether there is data indicating the processing target in 20 (
Step 2401). If data indicating the processing target exists, processing target state storage processing 2402 (21st
) (step 2402). This data does not exist when the current state storage instruction is given by the operator after changing the field of view of the camera (not shown) using the view control screen 104, and the state of the object instance camera is stored. do. That is,
The current state (position, direction, upward direction) of the camera (not shown) operated by the view control processing 129 is
The instance 2780 (FIG. 3c) for it in the object library 124 is looked up (step 2403). Then, in the command buffer 120,
Camera object instance 278 as processing target
0, stores the camera's current position, direction, and upward direction as arguments, and cameraset as the instruction code to form a command for the camera's field of view set, and sends this to the command queue at the top of the command queue stack]22. Add to the end (step 2404). Instruction code ca
m e r aS e t is an instruction code for a built-in instruction that places a camera (not shown) corresponding to camera object instance 2780 in a direction specified by an argument of the command.

On the other hand, as a result of the determination 2401, it is determined that the data indicating the processing target is present in the command buffer 120 because the operator uses the motion control screen 103 (FIG. 1) prior to instructing to store the current state. This is when an instruction to move or rotate an icon on the object display screen 102 is given by a mouse input. therefore,
When the processing target state storage processing 240 is started, the second
As shown in FIG. 1, whether the processing target indicated by the data in the command buffer 120 can be moved or rotated is checked from the attribute data regarding the object instance of the processing target (steps 2501 and 2504).

When determining movability or rotatability, “2.7
In the same manner as described in the motion control process 129j, objects that can be moved or rotated are selected from the object instance to be processed or the object instances that are its subordinate components as objects to be moved and objects to be rotated, respectively. If it is movable or rotatable, respectively.
The current position of the selected object instance to be moved and the current rotation angle of the object instance to be rotated are determined (step 2502). Then, for each movement or rotation, the object instance to be moved or rotated, the obtained current position or rotation angle, and a special instruction code related to the movement or rotation are stored in the command buffer 120.

or rotate to generate a command for movement or rotation. Each command is added to the end of the command queue at the head of command queue stack 122 (step 2502).

Through this process, commands for moving and rotating the icon specified by the operator can be embedded in the program.

4. Execution of object methods In this system, execution of object methods and programming of object methods can be performed at the same time. The system elements that realize this are described below.

4.1 Object method execution processing 135 (22nd
(Figure) As shown in Figure 22, when the object method execution process 135 is activated, it performs the following process in accordance with the command (command that is the target of this process).

(1) If the command is 5 success end, which instructs to end the program execution if the execution of a series of commands generated in advance is successful (step 2801), method execution end processing 240 (Figure 23) (Step 2802), reports successful completion (Step 2803), and returns control.

(2) If the command is not 5uccess end, check whether the code for the instruction of the command is a built-in instruction code (step 2804), and if it is for a built-in instruction, the built-in instruction is written to the instruction in advance. The program is executed using the determined program library (step 2806). If the execution is successful (step 2807), a successful completion is reported (step 2809) and the control is returned; if the execution is not successful, a failed completion is reported (step 2808) and the control is returned.

The calling command is not 5uccess end,
If the instruction code is not a built-in instruction code, the command has been defined by the user or will be defined in the future, and the following processing is performed.

(1) Check whether there is an unexecuted method among the methods already defined for the object instance specified by the processing target of the command (
Step 2811). If it does not exist, start the object method generation process 250 (step 2812),
Successful completion is reported (step 2813) and control is returned.

If it exists, it means that the user has defined the method, and we proceed to execute it.

(2) Select the first defined method among the unexecuted methods defined by the user for the object (step 2815). Because of this choice, when multiple methods are defined for each object,
Arrange them in the order of their definition and arrange them in the object instance library. Then, the object instance specified as an argument of the selected method is replaced with the object instance of the calling command (step 2816).

(3) Select the first command among the unexecuted commands in the command sequence that defines the method (step 2817), and execute the object method using this command as a calling command (step 2818)
.

(4) Check whether there are any unexecuted commands in the method (step 2819). If it does not exist, it means that the execution of the method has completed normally, and the process returns control by reporting successful completion (step 282).
1). If it still exists, the method has not completed yet, so proceed to the next step.

(5) Examine the results executed in (3) (step 282
0). If the execution is successful, the execution of the method is proceeding smoothly and the process returns to (3).

Otherwise, if the method has failed, there is a problem in the execution of the method, so return to (1) to recover from this problem.

4.2 Method execution end processing 2301 (Fig. 23) When the method execution end processing 2301 is started by the method execution processing 135 because the command to be executed is 5uccess, -end, as shown in Fig. 23, Perform the following processing according to the called command.

The called command is stored in the command queue stack 122
If the command corresponds to the command queue at the top of the command queue (step 2302), the following processing is performed. Otherwise, return control.

It is added as a command string in the command queue at the top of the command queue stack 122 to the instruction code method of the command to which the command is directed (step 2303).

Then pops the command queue stack 122,
Clears any object instances that were created while the popped command queue was at the top, and clears the method programming screen corresponding to the command. (Step 2304) Then, control is returned.

4.3 Object method generation process 2812 (second
(Figure 4) As shown in Figure 24, the following processing is performed.

A new command queue corresponding to the calling command is generated and this is placed on the command queue stack 121 (second
) (step 2910).

Additionally, a method programming screen corresponding to the method of the called command is generated (step 2921).
.

Then, the method name, the instruction target and the object name to which it belongs, and the argument and the object name to which it belongs are displayed on the screen (step 2930), and control is returned.

4.4 Built-in instruction execution processing 2806 FIG. 25 shows a built-in instruction list 3001.

In this diagram, object 3 is the target of the built-in instruction.
002. Required arguments 3003, built-in instructions 3004,
The effect 3004 when the command is executed is shown. The actual target or argument is an instance of an object or an atom.

C9 Operation Description The creation of a program according to this embodiment will be explained below using a specific example. The first example creates a program that makes phone calls. In the second example, a program is created that communicates with a large general-purpose computer and uses an existing application program on the computer.

In either case, a command for instructing the action to be performed by the person microcomputer 3102 is input as a program command, and this command is executed to cause the person icon 3102 to perform the action.

This command sequence is stored in the person object as one of the methods executed by the person object.

1. Creation of a program to make a phone call Here we will create a program (method) to make a phone call. The method name assigned to the person object as a method is call.

a. In the initial state of command input, the object display screen 102 is the 26th A
Suppose it is as shown in the figure.

The person icon 3102 (FIG. 1) represents the person object 2701 in FIG. 3A, and the telephone icon 3104
(Figure 2) represents the telephone object 2750 of Figure 3B. I haven't explained the phone number object yet, but
In the following, it is assumed that it has already been defined in the initial state. 3103 is an icon representing this object, which has a rectangular shape and has a slot value telephone number in its center. The instance name of the person object is rie.

The instance name of the telephone object is tel, and the instance name of the telephone number object is tn.

Below, these names will be used as names of icons representing the corresponding objects.

The person icon 3102 is an arbitrary phone icon 3
I am in a place far away from 104, and the phone icon 31o4
The receiver remains on the line.

And the number 01 is inside the phone number icon 31o3.
2-3456 is displayed. Further, as shown in FIG. 26B, the command queue stack 122 in the main memory 112 includes only an initial queue stack 1903.

Here, in order to have the person icon 3102 make a call, the user inputs the command (0) rie tel tn call. Here, everything other than call represents the processing target, and call
l is the method name.

(a-1) Below the command processing target and argument input,
``Processing target section'' refers to the processing target section of a command, unless otherwise specified or unless it is unnatural in the context.

To input this command, the operator uses the voice input device 111, the mouse 109, or the keyboard 111 to input the processing target PI, i, C13, and "t" as character strings.
Enter "e l" and "tn" in this order.

(a-1-1) In the case of voice input For example, to input a character string by voice, say ``Rie'' toward the voice input device 111 (FIG. 1). Voice is transmitted from the voice input device 111 to voice recognition processing 142 (second
Here, as shown in FIG. 7A, the closest candidate voice data is extracted from the voice data area of the candidate voice database 125 (step 601), and the voice event generation process 143 is activated (step 603). ).

Receive the extracted candidate audio data, confirm that the data type of the candidate audio data is an object instance name (step 701), and confirm that the event type is object selection and the additional information is the character string area II of the candidate audio data. Step 702) Input event processing 126 is activated to generate event data that is a copy of ri eI+ (Step 709). Input event processing 126 adds the received event data to incoming event queue 119 (FIG. 2).

Note that the above candidate voice data is created as follows when the person icon 31o2 is previously defined as an instance of the person object 2701. That is, at the time of this definition, the operator inputs the name of the instance using the keyboard, and at the same time, the system requests voice input of the name.

When the operator inputs this name by voice, it is stored as the content of candidate voice data for rie in the candidate voice database 125.

Event dispatch processing 127 (FIG. 2) can operate in parallel with operator input and processes event data in mechanical event queue 119. In the current example, r
Since the event type for event data for rieJ is object selection, object selection processing 130 (
2) is activated, and additional information "rrie" and a blank space are stored in the command buffer 120 (step 2001 (
Figure 14)).

(a-1-2) In the case of mouse input To input "rie'" using the mouse 109 (FIG. 1), move the mouse cursor 201 over the body of the person icon 3102 displayed on the object display screen 102. (
1) and press a mouse button (not shown). As a result, object selection event generation processing 30
4 is activated (Figure 8A). In this process 304, as shown in FIG. 8B, the position of the mouse cursor 201 and the three-dimensional shape database 123 (FIG. 2) are referred to (
It recognizes that the mouse button has been pressed on the person icon 3102) (step 8o1), and generates event data in which the event type is object selection and the additional information is the character string +1rie71 input from the keyboard.Step 802 ), the input event processing 126 is activated (step 803). In the input event processing 126, the sent event data is input to the input event queue 1.
Add to 19.

When this event data is later dispatched, as in the case of voice input, it is stored in the command buffer 120.
” and spaces are stored.

(a-1-3) In the case of keyboard input When inputting #rieI+ on the keyboard 110 (Fig. 1), place the mouse cursor 201 anywhere other than the object modeling screen 108 (Fig. 1), and then click r”
Type "i" and "e" as they are. Keyboard event processing 309 (FIG. 2) is activated in response to a burnt-out key. In this process 309, as shown in FIG. 9, after going through determinations 1301 and 1303, event data is generated in which the event type is the keyboard and the additional information is the key code given from the keyboard.Step 1304), input event processing 126 (step 1306). Input event processing 126 adds this event to incoming event queue 119.

When event data for such an input is dispatched later, the key codes therein are inputted as they are into the empty areas of the command buffer 120 in order from the front. In the present example, the command buffer 120 receives event data for each of "r", "i", and "e".
The processing target name "rie" is stored in .

In this way, in order to enable command input using voice input, mouse input, or keyboard input, each input from each is converted into data in a common format called event data.

Through the operations up to this point, the processing target +1rie II that should be included in the command to be input has been stored. Next, the processing targets II tel II and II tnI+ are added to the command. This operation is performed in the same way as the previous inputs. Regarding object selection, the same applies to subsequent command inputs. This is it,
The command buffer 120 contains rrie tel
tuJ is stored.

(a-2) Input of method name Subsequently, the name call of the newly defined method is input from the keyboard 110. Since this is the first method used in the system, this input is
It is possible to input only from 10 onwards. In response to this input, event data is generated and stored in incoming event queue 119 in the same manner as described above. Subsequently, in order to input the end of the input of one command, when a line feed code is input manually as a command input instruction, the keyboard event processing 141 executes step 1303 in FIG.
Event data whose event type is command input and whose additional information is empty is generated and added to the event queue 119. Since the event type of this event data is a command input, the event dispatch process 127 starts the command input process 131 (step 1405 (FIG. 13)). In this process 131, the contents of the command buffer 120 are added to the initial command queue 1903 at the top of the command queue stack 122, and the command buffer 120 is cleared (step 2102.degree. 2102 (FIG. 15)). The (0)rie tel tn call has now been sent to the initial command queue 1903. In this way, the command buffer 120 stores the entered character strings temporarily, and when the user subsequently gives command input instructions, the previously entered character strings are used as one command. There is.

Note that in this example, the method name was entered using the keyboard, but the method name can be entered using the candidate voice data 50.
If 1 is memorized in advance as shown below, input can be made using an undefined method or by voice. That is, the voice that indicates this method name is stored in the voice data 502 of the candidate voice data 501, the object method is stored in the data type 503, and "call" is stored in the character string 504. In this way, if the method call is voice input following the argument of the command (○), the candidate voice data extracted in the voice recognition process (Figure 7A) will be transferred to the voice event generation process (Figure 7B). , an event whose event type is command input is generated. The event generated is equivalent to the event generated by the keyboard method virtuoso keystroke 9call ten new line code'' described above.As already mentioned.

When the command processing is the target or argument, the type of data in the candidate data for the input voice is an object instance, so in this case, the input voice is stored in the command buffer. However, as mentioned above, voice input representing a method can be treated as the end of the command to be input, so candidate data for voice input can be treated as candidate data for voice that has already been input, and candidate data for voice that has already been input is It can be combined with data to form a single command.

In this way, if the type of voice input by the operator is determined to determine whether or not the input of the command is complete, the operator does not need to use special input such as a line feed code. The above voice input can of course be applied as is to predefined methods.

Note that the initial command queue 1903 is
The string stored in the program control screen 10
6 along with the command number (FIG. 26A).

In the following, it is assumed that command input is performed using the method described above.

b. The operator who executes the command uses the program control screen 106 (Fig. 1)
Mouse cursor 201 on top of “Run” button 215
, and press the mouse button (not shown) here. The event dispatch process 127 starts the program control process 132 (Fig. 2) for the event data generated by this operation, and during the execution of the process 132, the object method execution process 135 is started (step 2203 (Fig. 2)). Figure 18)). Here, the command queue 121 at the top of the command queue stack 122 is the initial command queue 1903 (26th
Figure B), and the first command is the above command (Figure B).
0), and the method call is undefined. However, the system invokes the object method execution outside R1A 135 that executes the command regardless of whether the method in the command is a predefined one.

In the process 135, the number is set as shown in FIG.

The instruction code is neither 5uccess end (step 2801) nor a built-in instruction (step 2804).
, and since it has not yet been registered as a method for the instruction object person (step 2811), the object method generation process 2812 is started (step 281).
2).

In this process 2812, as shown in FIG.
Area matrix for storing command sequences to be included in l)1
903 and command waiting area queue stack 1
22 (step 2910). In addition, a corresponding method programming screen 3110 is generated (Step 2920), and on the object method programming screen, the method name call, the set of the referent and its belonging object rie [person], the set of argument and its belonging object tel [telephone] are displayed.

tn [telephone number] is displayed (step 2930),
This is preparation for defining a new method.

From this point on, the operator inputs and executes each of the commands that make up the method call.

Here, (1) Go to the telephone (rie tel#positiongoto) (2) Pick up the telephone (rie tel tel take) (3) Make a call to the given telephone number.

(rie tel tn connect) (4
) If you can do the above, you will be successful.

(success end) Program something.

The operator inputs commands (1) to (4) in sequence. Here, # is for expressing the slot value of the object instance. Commands (1)-(4) are ca
The commands are added to the ll command queue 3121 in order. ca
The commands stored in the method queue 3121 are displayed on the corresponding method programming screen 31 each time they are stored.
10 is displayed.

Click "Run" on 106 on the program control screen.
Mouse cursor 20 is placed on button 215 (Figure 26A).
1 and press the mouse button, the object method execution process 135 executes the command (1) at the head of the command queue 3121 corresponding to the call method.
Execute.

Here, since the method goto is not 5uccesssend (step 2801), is not a built-in command (step 2804), and is not defined in the person object (step 2811), object method generation processing 2812 is performed. In other words, call method programming is temporarily interrupted and goto method programming is performed. For this purpose, a goto command queue 3020 (FIG. 27B) is created at the top of the command queue stack 122 during this process 2812 (step 2910 (FIG. 24)).

Further, on the display device 101, there is a message shown in FIG.
Method programming screen 3210 for to method
Method programming screen 3 for call method
110 (step 2910), the method name, processing target, arguments, etc. are displayed on the programming screen 3210 according to this command (1).
10 (step 2930). Thus, the queues in the command queue area queue stack 122 are equivalent to the number of method programming screens and their corresponding methods, and have the role of controlling method programming.

The goto method programs a person to move to a certain position. Here, enter the following command string.

(10) Create a temporary object instance that stores the distance dist 5calar@create (11) Create a temporary object instance that stores the direction dir vector@create (12) Check the distance between rie and tel rie te1
4position dist get distance
e(13) Check the direction of tel from rie
tel#takeposiitondirgetd
Direction (14) Directs rie in the direction of tel based on the results of (13).

rie dir rotate (15) Make rie walk the distance of (1)
dist walk (16) point rie in the direction of the phone
l#direction rotate (17) If the above is successful, you got it.

Terminate the method.

5uccess end commands (10) and (11) create a distance object that stores the distance dist (this is a scalar quantity) between the person icon 3102 and the telephone icon 3104, and the direction dir of the phone as seen from the person (this is a vector quantity) This is a command that creates direction objects that store each.

In this embodiment, the command for creating an object is (object name) @create, which is a built-in command. In FIG. 27A, icons 3202 and 3203 represent a distance object and a direction object, respectively, and these icons are displayed according to the definitions of these objects. Note that when objects are generated in this way, the names (dist, dir) of each object are input by voice. and,
The voice input to the voice data of candidate voice data.

An object instance is stored as the data type, and a character string ("dist"dir") representing the name is stored as the character string data, and stored in the candidate voice database.

command (12) (13), command (10)
The above distance dist,
Stores the value of direction dir. Then, with a command (14), the person icon 3102 is rotated in the direction dir, and with a command (15), it is made to walk a distance dist. Next, at step (16), make the user face the phone. Finally, the command (
17) Add (success end).

The above command sequence is g after inputting the last command (17).
It is stored in the oto command string area 3230. Next, when execution is instructed, the program execution processing 135 (Figure 22) executes this command queue one after another, and when the execution progresses until there is no factor causing failure <5 success end (step 2801), the goto method is Perform method execution termination processing 2301 (step 28
02). In method execution end processing 2301.

As shown in FIG. 23, the top of the command queue stack 122 is first examined (step 2302). Since this is the goto command queue 3230, the command string in this is stored as a goto method (
Step 2303). Then, the command queue stack 122 is popped and the got'o method programming screen 3210 is cleared. Also, got.

dist32, which is temporary data generated by method description
02, dir3203 to object library 124
．． It is deleted from the three-dimensional shape database 123 and object display screen 102 (step 2304).

At this point, the top of the command queue stack is cal
Return to l command queue 3121.

The remaining commands in the call command queue 3121 are (2) rie tel tel take (3)
rie tel tn connect(4)s
Success end. That is, from here on - the cal that was suspended
l method programming will be restarted.

Next, when execution is instructed again, the first command (2) in the call command queue 3121 is executed. tel
Since the take method is not defined in the instruction object human, object method generation processing 2
812 is performed. That is, call method programming is interrupted and tel take method programming is performed again.

As a result of this process 2812, command queue stack 1
At the top of 22, there is a tel number as shown in Figure 28A.
There is a take command queue 3320, and the method programming screen has a call command queue 3320 as shown in Figure 28B.
There is a screen 3110 for the method and a screen 3301 for the tel take method.

As a result of the previous execution of the goto method, person 3102 is standing in a position suitable for picking up telephone 3104 and facing the direction of the telephone. Therefore, here, an action is programmed in which the right hand of the person 3102 is extended toward the receiver of the telephone 3104, the receiver is picked up, and the right hand that has picked up the receiver is brought near the right ear of the person 3102.

To shape the upper right arm and lower right arm of the person 3102 into a shape that looks like a receiver of the telephone 3104, first, the upper right arm and the lower right arm are alternately selected and motion control processing 128 is performed on them. Here, the rotation valuator 220 around the X axis on the motion control screen (Fig. 1),
Rotation around axis valuator 221. The rotation valuator 222 around the z-axis is operated. As shown in FIG. 3A, the upper right arm and lower right arm have a rotatable attribute, so the right shoulder and right shoulder portions can be rotated. Also, both valves do not have the movable attribute, and if you try to move them, as you can see from the figure, the entire person will move, so they will not be moved. In this way, by skillfully controlling the upper right arm and lower right arm,
The motion control process 128 shown in FIG. 6 places the right hand near the telephone receiver. The reason why attributes are given in this way is to express that the joints of humans and the like can only rotate and cannot move.

After the above motion control'rie.

ruarm”, stores it as a processing target in the command buffer 120, and displays it on the program control screen 106.
By moving the mouse cursor 201 over the "memory" button 216 above and pressing the mouse button 202, the program control 132 is activated, during which step 2204 (step 2204) the state storage process 134 is activated. Figure 18). In the state storage process 134, the processing target (rie, ruarm) is the command buffer 12.
Since it is within 0, the storage process 210 is activated for the processing target (step 2402 (FIG. 20)).

In this process, the processing target in the command buffer is person 3.
102, and since this has only the rotatable attribute, the 0th order command (20) rie, ruarm rotates the right upper arm to the currently indicated rotation angle.
The current rotation angle otate is generated and the tel take command queue 33
20 (steps 25o4 to 2506). Similarly, by performing processing target state storage processing 210 on the lower right arm of the person 3102, the next command (21) rie, rlarm current rotation angle otate is added to the tel take command queue 3320.

Next, the receiver of te13104 is held in the right hand of rie3102. The built-in instruction 1ink is used for this purpose. The command is (22) rie, rhand tel.

It is receiver 1ink. This method 1ink is realized by specifying an argument object in the lower structure of the referent. As a result, from now on, the receiver moves and rotates according to the position of the person's 3102 right hand.

Further, even if the telephone 3104 is moved or rotated, the handset is not affected by the movement or rotation.

If this continues, the handset will move according to the movement of the telephone 31o4, so use the command (23) tel, body tel to break this relationship.

receiver unlink Execute.

As a result, the object structure of the person 3102 and the telephone 3104 becomes as shown in FIG. 28A.

Next, move the upper right arm and lower right arm of the person 3102 again to the valuator rot x on the motion control screen 103.
(220), rot y (221) + rot
z (222) to position the receiver so that it is positioned just near the right ear, and then the state storage process 134 is activated.

As a result, the command (24) rie, rarm current rotation angle otate (25) rie, rlarm current rotation angle otate is added to the tel take command queue 3320 .

As the last command for this method.

(26) Add success end. Then, these commands are executed one after another, and when the 5th success end is reached, method execution end processing 2301 is performed.

The command string written in the tel take method (
20)-(26), pops the command queue stack 122, and clears the tel take method programming screen 3301.

The next command in the ca11 command queue 3121 is (3) rie tel tn connect. The connect method is also a human object 270
1 (FIG. 3A), a new command queue 3430 is created, voice registration is performed, and object method generation processing 2812 is performed.

As a result, a connect command queue 3430 is created at the top of the command queue stack 122 (
Figure 29B), the method programming screen is called
Two items are displayed: one 3110 corresponding to , and one 3420 corresponding to connect (Fig. 29A).

Here, (30) Send a connection request to the specified phone number.
tn connectrequest (31) If the connection is successful, output a voice to that effect. rie "Connected" peak (32) If the above steps are successful, the process ends. 5uccess end.

Execute this command sequence, (32) success
When reaching s end, method execution end processing 2301 is performed, command strings (30) to (32) are stored as the connect method, the command queue stack 122 is popped, and the connect method programming screen 3420 is erased. .

Here, the top of the command queue stack is the call command queue 3121, and the top command is (4) success end. When this is executed, method execution termination processing 2301 is started, and (1)
-(4) Store the command string as a call method,
Pop the command queue stack 122 and call
When the method programming screen 3110 is cleared 6 or more, programming of the originally intended call method is completed. Also, the top of the command queue stack 122 here is the initial command queue 312.
It is 0. This completes the call method programming.

From now on, if you input "Execute" and send the command "Person object instance Telephone object instance enter Telephone number object instance callJ" to the object method execution processing 135, the person object instance will move to the position of the telephone and turn around. The user then picks up the receiver, issues a connection request to the specified phone number, and if the connection is successful, a voice saying ``Connected'' will be emitted.

Also, "Person object instance Vector object instance gotoJ.

The following commands are defined in the process of defining the call method: "Person object instance Telephone object instance tel takeJ", "Person object instance Telephone object instance Telephone number object instance C0nnectJ". Command execution is the action of the requested object instance. It is easy to recognize the state because it is output as a message or as a voice.

However, there is a possibility that a failure will occur somewhere during the execution of the command. In this example, the next command that requests a telephone connection is (30) rie tel tn connect in (3) rie tel tn connect.
It is conceivable that request may fail due to reasons such as the other party is busy talking. However, one of the features of the present invention is that there is no need to write a program until a failure actually occurs.

While executing the command several times, the command (30
), you can write a program using the failed example. In the example of FIG. 29A, it is assumed that one person 3102 uses a telephone and fails when using the telephone number object instance tnl. The command is (40) rie tel tnl connect and what fails is the command (30)' rie tel tnlconne which replaces command (30) with the current example.
ct request.

The object method generation process 2812 (FIG. 24) of command (4o) is performed in the same way as in the previous case.

The program here is for when telephone connection fails. The command sequence here is as follows.

(41) rie outputs a voice message saying “Failed” “Connection failed” peak (42) rie rotates the handset to fit into the main body
, receiver The direction in which the handset fits into the telephone
rotate (43) Place the handset at the main body position, rie, recexver tel#pos1txon translate (44) Separate the handset from rie's right hand, rie
cexver tel.

body 1ink (45) Return the receiver and main body to their original relationship rie, rh
and rie.

receiver unlink (46) Stand upright rxe neutral (47) If the above is successful, end 5uccess end shall be.

Now, if you execute the commands (44) and (45),
The structure of the object is as shown in Figure 30C 3330. This is the state initially defined for the person object 27o1 and the telephone object 2750. From this point on, the person and the phone operate as a normal person object 2701 and phone object 2750.

Execute the above commands one after another (47) success
When the end command is reached, method execution termination processing 2301 is performed, and the command sequence (41) to (47) is newly stored as the connect method of the person object 2701 without losing what was defined in FIG. 29B. Ru.

Now, two command strings (30)-(32) and command strings (41)-(47) are defined as connect methods for the person object. Since method execution is performed in the order of definition, when the command ``Person object instance Telephone object instance Telephone number instance C0nnec, tJ'' is executed, a specified connection request is issued using the command sequence defined in (30) to (32). As a result, if successful,
"Successful" is output. If you don't succeed,
Proceed to execute steps (41) to (47), output audio to J (r connection failed), hang up the receiver, and stand upright.

The secretary now has a method for making telephone calls. As shown in FIG. 3A, the person object 2701 uses a call method (2712)
, goto method (2713), tel take
Method (2714) + connect (1) Method (2715), connect
(2) It has a method (2716) and one among the group of built-in instruction codes shown in FIG. 25 whose processing target is a person.

2. Program using communication with large general-purpose computer 116 Next, features of the embodiment of the present invention will be explained using an example of programming to reserve a conference room using a large general-purpose computer. It also refers to programming the camera's field of view.

Person object instance rie (3102), terminal object instance term
(3612), 84-inch object instance d
ate3610, room object instance r o
Write the method yoyaku using om3611.

Here, it is assumed that the object instances are arranged in a three-dimensional space as shown in FIG.

Initially, the camera object instance is view 1 (
37Ql), and the terminal object instance term (3612) is not displayed on the object display screen because it is sandwiched between partitions 3602, as shown in object display screen 1 (3601) in FIG. 31A.

Therefore, it is assumed that this cannot be selected with the mouse 109 either.

method (50) rie term date roor
Let's say noyaku. Among these, term (3612) is input by voice or keyboard.

When you execute this, object method generation process 28
12 is started and y is placed on the command queue stack 122.
The oyaku command queue 3630 is booted and
A method programming screen 3620 is generated.

Here, if the terminal is not actually displayed, you will not know what you are doing, so operate the valuator on the view control screen 104 to change the term.
View 2 (3702) is the view where 3612 is visible
Change to Then object display screen 2 (36
If term 3612 is already displayed as in 02), it can also be selected using the mouse 109.

In this way, when selecting something that is defined as a three-dimensional shape in the computer using a display device and a mouse, if there is something that is not projected on the display device, you can change the view and select it on the display device. You can then select it.

This program executes state storage processing 134 for the camera, assuming that the operator wants to work in this view. To do this, it is sufficient to empty the command buffer for processing. When the state storage process 134 is executed, (51) a camera set command is generated for the position in which the camera term is visible and the direction in which the term is visible. This is a special command for changing views.

Next, using the predefined method goto, (52) r
je term#positiongot.

Add command to change rie3102 to term3612
Walk to the location and move it.

The instruction code for the next command (53) term openterminal is a built-in instruction that opens a communication path with a large general-purpose machine.
Log on to this. At the same time, a terminal screen 107 is displayed on the display device 101.

Next, command (54) term date room get
Execute reservexon to run the reservation application program on the large general-purpose computer, and save date3610 and room361.
Make a reservation for a conference room according to step 1.

If the above is successful, the command (55) success end is added to the yoyaku command queue 3630. If you execute this, the command sequence (51)-(55) will be yo
It is stored as a yaku method, the command queue stack 122 is popped, and the yoyaku method programming screen 3620 is cleared.

In this way, you can set a particular view position and specify that the camera should be placed at that position and an object method executed. They can also perform work using large general-purpose computers.

D. Modified example Up until now, the three-dimensional shape database 123 has been used to
Although an example has been described in which a dimensional shape is displayed on the object display screen 102, this is not necessarily essential to the present invention. Here, a method for realizing a system that does not use three-dimensional shapes will be described. For example, 1. This section describes the program that makes phone calls, which was explained in .

The problem with implementing a program to make a telephone call in a two-dimensional system is that the object instance that received the request goes to the telephone object instance,
It's about how to express having a phone. As shown in FIG. 33, this involves moving the object instance 3801 that received the request to the object instance 3802 that is the object of processing, and then placing an icon representing the phone to indicate that the person has picked up the phone. Emphasis processing such as blinking may be performed. Alternatively, both may be emphasized without any movement. Also, the 34th
As shown in the figure, the display of the requested object instance icon 3901 and the processing target icon 3902 may be changed. Furthermore, as shown in FIG. 35, from the requested icon 4001 to the icon 4002 to be processed,
An arrow may be displayed pointing toward the icon 4001 that received the request from 002.

As described above, the essential point of the present invention is that when the execution state of a program is expressed on a display device, the program that manages the execution of a program library is displayed graphically, and this is displayed according to the execution state. It is to change the.

Furthermore, in this embodiment, to make the person icon 3102 pick up the phone, command (2) is input, and this command is defined by a method consisting of command strings (20) to (26). However, for the purpose of notifying the operator of the execution status of the program, it is possible to embed the command strings (20) to (26) in advance in the method of the command connect of command (3) to make a call. It is also possible to have the person 3102 move toward the telephone set and pick up the telephone receiver near the operation of making a telephone call in response to the input of command (3).

Furthermore, in this embodiment, the handset is moved by linking to the person icon 3102, but the command (3) con
nect, the handset can be moved. This also allows the operator to know the methods of other objects requested by the program being created.

[Effects of the Invention] The display of the icon corresponding to the co-program changes depending on the stage of execution of the program and in relation to the icons representing other processing targets, so the user can intuitively grasp the execution status of the program. be able to.

Also, if the icon corresponding to the program is in the shape of a person, the user may see a secretary inside the computer.
You can instruct the execution of a program as if you were requesting a job.

Furthermore, if icons that are generally used for computer-based computers are made three-dimensional, an interface that is easier to intuitively understand can be realized.

In addition, it is possible to provide a computer operating environment that is easy to operate, especially when voice input/output is used for general computer input.

[Brief explanation of drawings]

FIG. 1 is an overall configuration diagram of a system that implements the present invention. FIG. 2 shows functional blocks of a program for program creation held in the main storage device of FIG. 1 and a database related thereto. 3A to 3C are diagrams showing various object instance information used in this embodiment, FIG. 4 is a diagram showing an example of graphic data in the three-dimensional database shown in FIG. 2, and FIG. FIG. 3 is an explanatory diagram regarding the structure and processing of candidate voice data in the voice database of FIG. 2; FIG. 6 is an explanatory diagram of the structure of event data processed by the program of FIG. 2. 7th
The figure is a flowchart regarding voice input processing. FIG. 8 is a flowchart regarding mouse input processing. FIG. 9 is a flowchart regarding keyboard input. Figure 10 is a command format diagram, Figure 11 is a command queue structure diagram, Figure 12 is a command queue staff structure diagram,
FIG. 3 is a flow diagram of an event dispatch process that interprets user input. 14 to 19 are 1. 3 is a flowchart regarding processing initiated by event dispatch processing. FIGS. 20 and 21 relate to the state storage process that is activated by the program control process. This is a flowchart. 22 to 24 are called during object method execution processing. 3 is a flowchart regarding method execution termination processing. FIG. 25 is a diagram showing a group of built-in instructions used in the program of FIG. 2. 26 to 3 (Each figure is an explanatory diagram of an embodiment using a specific example, and explains the state on the display device and the main state on the main storage device. FIG. 32 33 to 3 are diagrams explaining the arrangement of object instances in three-dimensional space and the views that project them.
FIG. 5 is a diagram illustrating a method for easily realizing the present invention in a two-dimensional system. Explanation of symbols 135--One object method execution process, 123--
3D shape database, 124--Object library, 122--Command queue stack, 125
- one candidate speech database, 142- one speech recognition processing,
143-1 Audio event generation processing. Dye 3△ Kawanere Sukeban Otsuko is equipped with "L 773 Tsu'u 3C circle ζmu camera off', /" 1. B field entry event p processing G 萱正D ru 8 △ proverbial construction wa] 1st 辣口■ /ρ fig.
L, Emperor hko [puff's 17th meso-11-''bird IIl second part 4 boots (J] u / no ra / 2/ no. ;'vFu1+kuoff columnToha#tgmethodanfactorystrategy15commandinputtheoremNo.23Figure 73Noah'rJ7-'71-j Seven tapers 1 lure evening ξ work! 210 μm 1 skewer 741 rules V 22u 13 13 offre 1 1 men l de ° large Kjfi JLK q 27A 27 BTA 22 zu cloak fight 15 lines sosu 2- /2 2nd Epidemic Map $ x'lya Figure 30A Figure 32

Claims (1)

[Claims] 1. Displaying a plurality of icons each representing one of a plurality of processing targets, each of which has at least one process to be performed, and an icon assigned to a program to be created; In a program creation method, the program is created by inputting a sequence of commands for configuring a program that executes a desired process including at least one process determined for at least one of the plurality of processing targets. , one process target for which the one process is determined in response to one predetermined command that is input near the time when the first command requesting execution of the one process is input. A program creation method comprising the steps of changing the display of the assigned icon so that the one icon representing the icon can be identified. 2. Inputting the command string includes the step of inputting at least one second command that requests a change in the display of the assigned icon near the time when the first command is input; The method for creating a program according to item 1, wherein the change is made in response to input of the second command. 3. The method for creating a program according to item 1, wherein the display change is performed in response to input of the first command. 4. The program creation method according to item 1, wherein the display change comprises the step of moving the assigned icon in the direction of the one icon. 5. The display modification includes a first step of transforming the assigned icon in relation to the one icon.
How to create a program in section. 6. The program creation method according to item 1, wherein the display change comprises the step of adding an arrow pointing to the one icon to the assigned icon. 7. Inputting the command sequence includes the step of inputting at least one second command that requests a change in the display of the assigned icon near the time when the first command is input; The method for creating a program according to items 4 to 6, wherein the change is made in response to the second command. 8. The method for creating a program according to items 4 to 6, wherein the display change is performed in response to the first command. 9. The method for creating a program according to item 1, wherein the assigned icon has the shape of a person. 10. The program creation method according to item 9, wherein the display change comprises the step of moving the icon having the shape of the person in the direction of the one icon. 11. The display change includes a first step of displaying the state of movement of the icon having the shape of the person as a moving image.
How to create a program for item 0. 12. The program creation method according to item 11, wherein the display change comprises the step of displaying an icon having the shape of the person so that the person walks. 13. Each of the plurality of icons has a shape obtained by projecting a three-dimensional object representing the corresponding processing object arranged in three-dimensional space onto the display screen, and the assigned icon It has a shape obtained by projecting a person placed in a three-dimensional space onto the display screen, and the display change involves changing at least one of the shape and position of the person in relation to the one icon. 1. The method for creating a program according to item 1, further comprising the step of displaying a figure obtained by projecting the person onto the display screen. 14. Inputting the command sequence includes a step of inputting a plurality of second commands for selectively moving a plurality of parts constituting the person in the vicinity of the time point at which the first command is input. and the display change comprises the step of selectively moving a plurality of parts constituting any one of the persons in response to the plurality of second commands. How to make. 15. The display change is performed in response to the first command,
Items 9 to 13 consisting of the step of changing the display.
How to create a program in section. 16. The method further comprises the step of executing each of the input commands and notifying the operator of the results, and the display change is performed in response to execution of the one command. One of the following program creation methods. 17. In response to execution of the first command, execute a predetermined program for executing the one process, and in response to a command in the predetermined program, the one icon changes to another. 17. The program creation method according to any one of paragraphs 1 to 16, further comprising the step of changing the display of the one icon so that the icon can be identified from the icon. 18. Displaying a plurality of icons each representing one of a plurality of processing targets, each of which has at least one process determined for the plurality of processing targets; In a program creation method that creates a program by inputting a sequence of commands to configure a program that executes a desired process including at least one process, inputting a first command requesting execution of the one process. A program creation method for changing the display of one icon so that the one icon representing one processing target for which the one processing is determined can be identified from other icons in response to the above. 19. Further steps of executing each of the input commands, notifying the operator of the results, and executing a predetermined program for executing the one process in response to the execution of the first command. and the display change changes the display of the one icon so that the one icon is distinguishable from other icons in response to execution of one command in the predetermined program. 18. The method for creating a program consisting of steps. 20. The method for creating a program according to item 18, wherein the display change comprises the step of changing the position of at least a portion of the one icon in response to execution of the one command in the predetermined program. 21. Displaying a plurality of icons each representing one of a plurality of objects to which at least one process to be performed is assigned, and an icon representing an object to which a program to be created is assigned; In an object-oriented program creation method, the program is created by inputting a sequence of commands for configuring a program that executes a desired method including at least one method defined for at least one method. In response to one command input in advance near the time when the first command requesting the execution of one method was input, one icon representing one object for which one method is defined is displayed. A method for creating a program comprising the steps of changing the display of an icon representing an object assigned to the program so that the object can be identified. 22. Inputting the command string is a step of inputting at least one second command that requests a change in the display of an icon representing an object assigned to the program near the time when the first command is input. 22. The program creation method according to claim 21, wherein the display is changed in response to input of the second command. 23. The program creation method according to item 21, wherein the display change is performed in response to input of the first command. 24. The program creation method according to item 21, wherein the display change comprises the step of moving an icon representing an object assigned to the program in the direction of the one icon. 25. The program creation method according to item 21, wherein the display change comprises the step of transforming an icon representing an object assigned to the program in relation to the one icon. 26. The program creation method according to item 21, wherein said display change comprises the step of adding an arrow pointing to said one icon to an icon representing an object assigned to said program. 27. The input of the command sequence is a step of inputting at least one second command that requests a change in the display of an icon representing an object assigned to the program near the time when the first command is input. The method for creating a program according to any one of paragraphs 24 to 26, wherein the display change is performed in response to the second command. 28. The display change is performed using the 24th command, which is the first command.
A method for creating a program according to any one of paragraphs 26 to 26. 29. The method for creating a program according to item 21, wherein the icon representing the object assigned to the program has the shape of a person. 30. The program creation method according to item 29, wherein the display change comprises the step of moving the icon having the shape of the person in the direction of the one icon. 31. The display change includes a third step of displaying the state of movement of the icon having the shape of the person as a moving image.
How to create a program for item 0. 32. The method for creating a program according to item 31, wherein the display change comprises the step of displaying an icon having the shape of the person so that the person walks. 33. Each of the plurality of icons has a shape obtained by projecting a three-dimensional object placed in a three-dimensional space and representing a corresponding processing target onto the display screen, and each icon has a shape obtained by projecting a three-dimensional object representing the corresponding processing target placed in a three-dimensional space, and has an object assigned to the program. The represented icon has a shape obtained by projecting a person placed in the three-dimensional space onto the display screen, and the display change involves changing at least one of the shape and position of the person to the one icon. 32. The method for creating a program according to item 31, comprising the step of displaying a graphic obtained by projecting the person after the relevant change onto the display screen. 34. Inputting the command sequence includes a step of inputting a plurality of second commands for selectively moving each of the plurality of parts constituting the person in the vicinity of the time point at which the first command is input. and the change of the display is performed by the program according to any one of paragraphs 29 to 33, which comprises the step of selectively moving a plurality of parts constituting the person in response to the plurality of second commands. How to make. 35. The display change comprises the step of changing the display in response to the first command.
How to create a program in any one of the sections. 36, further comprising the step of executing each of the input commands and notifying the operator of the results, wherein the display change is performed in response to the execution of the one command according to paragraphs 21 to 35. One of the program creation methods. 37. In response to execution of the first command, execute a predetermined program for executing the one method, and in response to a command within the predetermined program, the one icon changes to another. 37. The program creation method according to any one of paragraphs 21 to 36, further comprising the step of changing the display of the one icon so that the icon can be identified from the icon. 38. Displaying a plurality of icons each representing one of a plurality of objects, each of which has at least one method defined for each object, and at least one method defined for at least one of the plurality of objects. In a program creation method in which a program is created by inputting a sequence of commands to configure an object-oriented program that executes a desired method including two processes, the first command that requests execution of the one method is A program creation method for changing the display of one icon representing one object for which one method is defined in response to an input so that the one icon is distinguishable from other icons. 39, executing each of the input commands, notifying the operator of the results, and executing a predetermined program for executing the one process in response to the execution of the first command; further comprising: changing the display so that the one icon is distinguishable from other icons in response to execution of at least one predetermined command within the predetermined program; 39. The method for creating a program according to item 38, comprising the step of changing the display of two icons. 40. The program creation method according to item 18, wherein the display change comprises the step of changing the position of at least a portion of the one icon in response to execution of the one command in the predetermined program. 41. Display a plurality of icons each representing one of a plurality of processing targets, each of which has at least one process to be performed, and an icon assigned to a program to be created, In a program execution method for executing a program that executes a desired process including at least one process defined for at least one process, the method includes: requesting execution of the one process among the command strings forming the program; The assignment is made such that in response to execution of one predetermined command in the vicinity of the first command, one icon representing one processing target for which one processing is determined can be identified. A program execution method comprising the steps of changing the display of an icon. 42, the command string includes at least one second command that requests a change in the display of the assigned icon;
42. The program execution method according to item 41, wherein the program execution method is provided near the first command, and the display is changed in response to execution of the second command. 43. The program execution method according to item 41, wherein the display change is performed in response to execution of the first command. 44. The display change comprises the step of moving the assigned icon in the direction of the one icon.
How to run the program in section. 45. The program execution method according to item 41, wherein the display change comprises the step of transforming the assigned icon in relation to the one icon. 46. The program execution method according to item 41, wherein the display change comprises the step of adding an arrow pointing to the one icon to the assigned icon. 47. The command string has at least one second command near the first command that requests a change in the display of the assigned icon, and the display change is performed by executing the second command. The program execution method according to any one of paragraphs 44 to 46, which is carried out in response to. 48. The program execution method according to any one of items 44 to 46, wherein the display change is performed in response to execution of the first command. 49. The program execution method according to item 41, wherein the assigned icon has the shape of a person. 50. The display change is
49. The program execution method according to item 49, comprising the step of moving the icon having the shape of the person in the direction of the one icon. 51. The display change includes a fifth step of displaying the state of movement of the icon having the shape of the person as a moving image.
How to run the program in item 0. 52. The program execution method according to item 51, wherein the display change comprises the step of displaying an icon having the shape of the person so that the person walks. 53. Each of the plurality of icons has a shape obtained by projecting a three-dimensional object placed in a three-dimensional space and representing the corresponding processing target onto the display screen, and the assigned icon A person placed in a three-dimensional space has a shape obtained by projecting the person onto the display screen, and the display change includes changing at least one of the shape and position of the person in relation to the one icon. 42. The program execution method according to item 41, comprising the step of displaying a figure obtained by projecting the container onto the display screen. 54, the command string has a plurality of second commands for selectively moving each of the plurality of parts constituting the person in the vicinity of the first command, and the display change is performed by the plurality of parts constituting the person. 53. The program execution method according to any one of items 49 to 53, which comprises the step of selectively moving a plurality of parts constituting the person in response to execution of the second command. 55. The program execution method according to any one of items 49 to 53, wherein the display change comprises the step of changing the display in response to execution of the first command. 56. In response to the execution of the first command, execute a predetermined program for executing the one process, and in response to the execution of the command in the predetermined program, display the one icon. 56. The program execution method according to any one of paragraphs 51 to 55, further comprising the step of changing the display of the one icon so that the icon is distinguishable from other icons. 57. Displaying a plurality of icons each representing one of a plurality of processing targets, each of which has at least one process determined for the plurality of processing targets; In a program execution method for executing a program that executes a desired process including at least one process, in response to execution of a first command requesting execution of the one process, the one process for which the one process is specified is executed. A program execution method for changing the display of one icon so that the icon representing one processing object can be identified from other icons. 58, further comprising the step of executing a predetermined program for executing the one process in response to execution of the first command, and the display change is performed by executing one of the predetermined programs in the predetermined program. 58. The program execution method according to item 57, comprising the step of changing the display of the one icon so that the one icon is distinguishable from other icons in response to execution of the command. 59, displaying a plurality of icons each representing one of a plurality of objects to which at least one process to be performed is assigned, and an icon representing an object to which an object-oriented program to be created is assigned; A program execution method for executing an object-oriented program that executes a desired method including at least one method defined for at least one processing target, the method comprising: In response to the execution of one predetermined command near the first command that requests the execution of one process, one icon representing one object to which one method is assigned can be identified. A program execution method comprising the steps of changing the display of an icon representing an object assigned to the program so that the object is assigned to the program. 60, the command sequence has at least one second command near the first command that requests a change in the display of an icon representing an object assigned to the program, and the display change is performed by the first command; 59. The program execution method according to item 59, which is carried out in response to execution of the second command. 61. The program creation method according to item 59, wherein the display change is performed in response to execution of the first command. 62. The program execution method according to item 59, wherein the display change comprises the step of moving an icon representing an object assigned to the program in the direction of the one icon. 63. The program execution method according to item 59, wherein the display change comprises the step of transforming an icon representing an object assigned to the program in relation to the one icon. 64. The method of executing a program according to item 59, wherein said changing of the display comprises the step of adding an arrow pointing to said one icon to an icon representing an object assigned to said program. 65, the command sequence has at least one second command near the first command that requests a change in the display of an icon representing an object assigned to the program, and the change in the display includes 64. The program execution method according to paragraphs 60 to 64, which is carried out in response to execution of a second command. 66. The program execution method according to any one of items 60 to 64, wherein the display change is performed in response to execution of the first command. 67. The program execution method according to item 39, wherein the icon representing the object assigned to the program has the shape of a person. 68. The program execution method according to item 67, wherein the display change comprises the step of moving the icon having the shape of the person in the direction of the one icon. 69. The display change includes the step of displaying the state of movement of the icon having the shape of the person as a moving image.
How to run the program in Section 7. 70. Changing the display comprises the step of displaying an icon having the shape of the person so that the person walks
How to run the program in section. 71. Each of the plurality of icons has a shape obtained by projecting a three-dimensional object placed in a three-dimensional space and representing the corresponding processing target onto the display screen, and the assigned icon It has a shape obtained by projecting a person placed in a three-dimensional space onto the display screen, and the display change involves changing at least one of the shape and position of the person in relation to the one icon. 59. The program execution method according to item 59, comprising the step of displaying a figure obtained by projecting the person on the display screen. 72, the command string has a plurality of second commands for selectively moving each of the plurality of parts constituting the person in the vicinity of the first command, and the display change is performed by the plurality of parts. 72. The program execution method according to any one of items 67 to 71, comprising the step of selectively moving a plurality of parts constituting the person in response to execution of the second command. 73. The program execution method according to any one of items 67 to 71, wherein changing the display comprises changing the display in response to execution of the first command. 74. In response to execution of the first command, execute a predetermined program for executing the one method; and in response to execution of a predetermined command within the predetermined program, 74. The program execution method according to any one of paragraphs 59 to 73, further comprising the step of changing the display of one icon so that the one icon is distinguishable from other icons. 75, displaying a plurality of icons, each representing one of a plurality of objects, for which at least one process to be performed is determined; In a program execution method for executing an object program that executes a desired method including two methods, in response to execution of a first command requesting execution of the one method, one method for which the one method is defined is executed. A program execution method for changing the display of one icon representing an object so that the icon can be identified from other icons. 76, further comprising the step of executing a predetermined program for executing the one process in response to execution of the first command, and the display change is performed by executing a predetermined program in the predetermined program. 76. The program execution method according to item 75, comprising the step of changing the display of the one icon so that the one icon is distinguishable from other icons in response to execution of the command. 77. In a method for displaying a plurality of icons on a display device for selection by an operator, each of which is input to a computer as representing an object to be processed by a program, a plurality of three-dimensional objects arranged in a three-dimensional space. An icon display method that projects a figure visible when viewed from a first viewpoint onto a two-dimensional plane, and displays each of a plurality of figures obtained by the projection as one of the plurality of icons. 78. The icon display method according to item 77, wherein one of the plurality of three-dimensional objects has the shape of a person. 79. Change the viewpoint from which the plurality of three-dimensional objects are viewed to a second viewpoint, project the figures of the plurality of three-dimensional objects as seen from the second viewpoint onto a two-dimensional plane, and obtain the following: 78. The icon display method according to item 77, wherein each of the new plurality of figures is displayed as a new figure of the plurality of icons. 80. The icon display method according to item 79, wherein the viewpoint is changed substantially continuously. 81. The icon display method according to item 79, wherein the second viewpoint is selected so that a certain part of the plurality of three-dimensional objects that cannot be seen from the first viewpoint is visible. 82. The icon display method according to item 79, in which the viewpoint is changed to a point specified by the operator. 83. The icon display method according to any one of items 77 to 82, wherein the plurality of icons are icons used for creating a program. 84. When inputting a command consisting of a first part representing the processing content and at least one second part representing the processing target or argument by voice, the type of the input voice is either the first part or the second part. If it is the second part, it is stored as part of the command to be input, and if it is the first part, the first part is added to the already stored second part. A method for inputting commands by voice, characterized in that the commands are combined and used as if the input of the commands to be inputted has been completed. 85. When a three-dimensional shape expressed in a computer is decomposed into parts and the parts are structured hierarchically, each part is given a movable attribute and a rotatable attribute, and this can be moved or rotated. A method for manipulating a three-dimensional shape, which allows the movement or rotation only when a movable attribute or a rotatable attribute is checked and the rotation is possible.