JP2003280928A - Information processing device by object network - Google Patents

Abstract

(57) [Summary] An object of the present invention is to realize a system which facilitates development of an application having visibility, interactivity, and coordination with respect to an information processing apparatus using an object network. SOLUTION: In a well system 20, a system hierarchy is defined by a relationship between a data model, an object model, and a process model, and the model hierarchy is made to correspond to the system hierarchy. Treat data and constraints equally in the system and unify the flow of data and constraint processing. System constraints are defined by inter-object functions in the object network, thereby controlling the way constraints are handled.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device that handles data and processing on the data as an object and processes information by an object network that expresses them in a graph. The present invention relates to an information processing device based on an object network that is defined by an inter-object relationship and controls the constraint processing method accordingly.

2. Description of the Related Art For example, by modeling a system of a specific application field such as an image processing system as an object network, a system developer can easily construct a system of the specific application field, and Japanese Patent Laid-Open No. 5-233690 (Language Processing System by Object Network) is a processing system that enables a user of a system of a specified application field to grasp the system state and executable operations in an easy-to-understand manner. Known by.

In this processing system, objects are roughly classified into noun objects and verb objects, an object network in which noun objects are expressed as nodes and verb objects are expressed as branches is used as a reference model. It is intended to provide an information processing mechanism such that, when an existing noun object is actuated by a content of a function as a verb object, a target noun object in the direction of a branch corresponding to the verb object name is obtained.

FIG. 29 shows an example of an object network for image drawing processing. None, Poin in Figure 29
t, Point seq., Line,… are noun objects, and d
efine, draw up, ... are verb objects. For example, consider the case where an image is drawn on the display screen. Initially, there is nothing, and the point is drawn by designating the point with the mouse or the like. Further, some points are designated, and the state transits to a state in which a point sequence is formed.

When such data such as points and point sequences is regarded as a noun object, a verb object (for example, define) as a function operates on a certain noun object (for example, Point), so that the target noun object is obtained. (Eg Point seq.) Is obtained. That is, each state of data to be drawn is placed in a node as a noun object, and a branch connecting the nodes is set as a verb object, as shown in FIG.
If a network is expressed, the structure of the image processing system for drawing can be formed as it is by the object network.

Further, in order to effectively advance the processing corresponding to the object network by the interaction between the client and the server, the visualization target item corresponding to the object network is displayed, and the instruction and data intended by the client are displayed. A common platform with various windows for displaying the results executed by the server is provided, and the communication manager in the common platform intervenes the windows to match the data representation format between the client and server. Commons that are designed to mediate
An interactive information processing device having a platform function (Japanese Patent Laid-Open No. 7-295929) is known.

DISCLOSURE OF THE INVENTION The present invention is directed to a language processing system using an object network disclosed in Japanese Patent Laid-Open No. 5-233690 and Japanese Patent Laid-Open No. 7-2.
It is an object of the present invention to further expand and develop the function of the interactive information processing device based on the common platform function disclosed in Japanese Patent Publication No. 95929 to provide a system in which it is easy to develop an application having visualization, interactivity, and cooperation. And Specifically, by relating the constraint processing to the model concept, we clarify the rules for data and constraints and the software structure, clarify the rules between objects in the target field, and then create the object relationships. The object is to provide a means for performing a coupling process with a power process.

FIG. 1 shows a structural example of the present invention for solving the above problems. In FIG. 1, 10
Is a user (client) that gives data and instructions to this system, and 20, 21, and 22 treat data and processing for the data as objects, and process information by an object network that graphically represents them (WELL: Window). -based Ela
Well system based on the language system of boration language (30, 31, 32) is a common platform for providing an interface as a place of interaction between the well system and a user or another system, 40, 41, 42.
Represents an operation control system for controlling a process based on the object network, and 50, 51 and 52 represent a data management system for managing a data group related to the object network.

In particular, the well system 20 is used by the user 1
0 is an agent role server having an agent function of preparing a service plan according to the intention of 0, and the well systems 21 and 22 are the specific role servers that play a role of executing the service plan prepared by the agent role server. Is. The data management system 50 represents a data model in which the attribute value structure is expressed by using a template format schema, and hierarchically divides each object into a format model and a feature model based on the template format of the data model and represents the model. Object model, and graph representation with noun objects as nodes and verb objects as branches,
When the contents of a function as a verb object are activated on a noun object existing at a node, an object network is obtained in which the target noun object in the direction of the branch corresponding to the action object name is obtained. The data group that defines the system hierarchy is managed by the relationship between corresponding process models.

The operation control system 40 has a control means for connecting the models by checking the constraints given to the models on the basis of the data group managed by the data management system 50 and advancing the process. The same applies to the data management systems 51 and 52 and the operation control systems 41 and 42 in the well systems 21 and 22. However, the well systems 21 and 22 operate as a specific role server that performs processing based on a specialized object network corresponding to individual services.

The operation control system 40 processes the data and the constraint by the same processing method in the system, and the user 10 for the data and the specific role server (well system 21, 22) for the constraint.
First, a service is requested, and the process proceeds with data or constraints obtained by the response operation to the request.

Further, the operation control system 40 defines the constraint by the relation between objects in the object network, and controls the constraint processing method according to the constraint. In particular, as these constraints, there is a constraint between attribute values of noun objects, that is, an AND constraint that indicates that it cannot move to the next state unless all required attribute values are available, and between verb objects that start from a certain noun object. The attribute value set of the upper object at the present time is calculated from the relationship between the upper object and the lower object in the processing procedure in the object network, and the OR constraint indicating that they have independence and can be executed in parallel. , There is a hierarchical constraint indicating that a new operation of a lower object receives a matching operation.
Controls process execution based on these constraints.

When designing an object network for service execution, the user 10 as an expert has means for displaying information about constraints based on the model of the system and making those constraints available. It is possible to specify a process model for performing an execution process based on a design relating to the generated constraint by referring to the displayed constraint.

It is possible to configure a plurality of agent role servers such as the well system 20 in a hierarchized form. In this case, the upper agent
The role server can regard the subordinate agent roll server as a specific role server, and the agent role server can be configured to control the specific role server depending on the hierarchical structure of the model.

BEST MODE FOR CARRYING OUT THE INVENTION First, the concept of an object handled by the system of the present invention will be described. FIG. 2 is a diagram showing the connectivity between the real world and the emotional target world. In the real world, where humans face each other, a large number of objects are related to each other or move almost independently, and in this target world, humans exert their five senses to express new interest as interest, recall, imagination, and dreams. To do. By this action, as shown in FIG. 2, a small part of the real world is mapped to the sensitive target world. It differs from the real world in that it is virtual and can only be perceived, referenced, or indicated as an object. By increasing the level of this cognitive refinement more logically, communication between humans becomes more dense, and cooperative assimilation and knowledge acquisition among groups of equal interest are possible. become.

In order to proceed with the above process, a target model and an objective calculation model are required. An object of the present invention is to provide a well system (having an expandability capable of coping with various object networks) disclosed in Japanese Unexamined Patent Publication No. 7-295929 (interactive information processing device with common platform function). (Sometimes called the Extensible Well System)
In this paper, by relating the constraint processing to the model concept, we explain the rules for data and constraints and the software structure, clarify the rules between objects in the target field, and then describe the object relationship and the process to be created. It is to provide a means for performing a joining process between the two. Here, the real world of the target field does not necessarily correspond to the sensuously perceived world, which is merely a virtual expression. Therefore, software creation needs to be easy to modify interactively.

The target model utilizes the object concept used in software. In the object,
Data and constraints can be treated in the same way. Less than,
Describe data and constraints. Data is mathematically expressed as an input argument and an output argument of a function, and in the system, a data name is given as an attribute name of a noun object and an attribute value is given as a value. In the system of the Well system, when entering the data definition process, a definition preparation process (definingprocess) that prepares a template corresponding to the noun object name, and a definition operation that sets the value at the corresponding position in the template (Define operati
on) and the stage is divided and handled. The verb object performs some function operation on the noun object. There are common nouns and proper nouns in the noun case, and the determiner {a, th
e} gives a unidirectionality for the reference instruction and the confirmation of the data value, and the genericity and the concreteness are determined.

The modification is performed with adjectives and adverbs, but the modification with respect to genericity has a partial order of modification due to constraint priority. For example, comparing featured and colored, the constraint priority is determined in this order. The fact that the object represented by the name α has the property represented by γ is written as a join support notation like γ {α} according to the notation of inclusive logic. Then, it shows that the object represented by α has the property represented by γ. Therefore, the set of predicates of P {j} defines the object {j}.

For example, Line segment ([Line se in FIG. 29
g.]) defines a Line in the form shown in the object network of FIG. 29, and when the drawing work is currently being performed, painted by {th
e identified Colored line} ＝ [the list of Colore
d line seg.] and appears in the corresponding work process
Colored line seg. Is required as the answer. Although the property of the constraint is defined by an inclusive equation, an element having an extensional property is considered as follows.

For example, synmetrical element {an identif
ied element} internally gives the property that the counterpart element should have with respect to the symmetry point / line, but externally, the corresponding element must be retrieved from the database. As described above, data is considered as a value of a specific attribute, and constraint is considered as a property of a generic object. As a representation of the processing system, the data is decided by the client and the constraint is decided as a service, and its contents are treated in a coordinated manner. That is, in terms of notation, the processing is performed as a difference between a small bracket and a curly bracket.

As a software process, as a language system that can be commonly specified by people involved in the process through requirements, specification design, implementation, and evaluation of software, an extended function language extensible well ( Extensible WELL, Japanese Patent Laid-Open No. 5-2 mentioned above
No. 33690, JP-A-7-295929). The feature of this language system is that it uses natural language words and sentences and displays an object network on a common platform as a common place between users and servers to visualize the relationships between words.
The operation is to perform a specific and purposeful calculation. In addition, the constraints are displayed in the form of an expression or simple sentence, and you do not have to create a program
It has the feature that the user can effectively execute the operation as the user intended.

In order to effectively use the above-mentioned language system and the cooperation function by interaction, the following model structure is provided. The classification and structure of the model will be described below. The nouns, verbs, and modifiers of adjectives and adverbs each have templates, and the templates for modifying the contents of words are connected to be integrated.
Therefore, the following model is set.

(1) Data model: Well (WEL
In the language system of L: Window-based Elaboration Language, the attribute value structure is expressed using a template format schema. Through this data model, the notation format of objects, data operations, etc. are managed, data is controlled, and each object is associated with a process.

Data cases are noun objects, verb objects, their modifier objects (adjectives,
Adverbs) are considered and associated with the known quasi-natural language form TELL.

(2) Object model: The object model is hierarchically divided and expressed into a formal model and a feature model according to the data model.

(A) Formal model: In the image, points, lines,
A three-dimensional structure, a motion structure, etc. from a region are made into a composite format using a template as a data model, and all pattern items are described in the format. A relationship between a plurality of simple objects, for example, a new object due to hiding, reflection, and refraction due to the context, is also described as a template and is modeled. That is, the formal model provides a syntactic representation. The formal model corresponds to the geometric model in the case of an image.

The formal model of a verb object has a noun object name to be inherited as a state constraint as a precondition for starting execution of data operation, a template part for instructing a constraint name, an incondition as a running constraint, Postcon as a constraint after execution ends
have a dition. These constraints are combined with the constraints indicated by being added to the noun object as a name to define the relationship from the generic object to the concrete object.

(B) Feature model: A feature is expressed based on the attribute value of a noun object, and it has some invariance with respect to the environment. An example thereof is a dividing line, and individual components such as eyes, nose, and mouth are arranged in areas separated by the dividing line of the face image. Highlight lines also have similar properties and play a major role in stereoscopic expression.

The motion feature model is mainly composed of a verb object for motion and a noun object as a motion target.

(C) Sensitivity model: The sensitivity model is a model for embedding human sensitivity in a formal model and a feature model, and a name and a modification parameter are designated as sensitivity words at any time so that they can be changed.

(3) Process model: A noun or verb object is expressed as a process, and is expressed as a graph as an object network. This design was done by an expert of the client, WELL
The system can be driven by the graph structure editor of.

Furthermore, the agent role server (ag
An ent role server) and a specific role server (specific role server) are required to have a model for coordinating and integrating processes, and the formal model defines the form of interaction between individual processes. Regarding the relationship between the agent role server and the specific role server,
As described in detail in JP-A-7-295929,
Only a brief explanation is given here.

Each service requests execution of inherited data and constraints by request and respond commands, and the combined data and constraints report execution results as execution results. It is formalized using the concept. Again, the following feature models are possible with the formal model above. The feature model is a model for adding a constraint function to the role function according to the environment, and is defined as an objective calculation model.

As described above, in the case of an image, a geometric model is an example of a formal model of an object. As models for constructing the geometric model, it is necessary to use for scalar data and for vector data. On top of that, a structural model for the object is constructed, but here, the region definition is performed for the individual elements constituting the object. For example, when it comes to human faces,
There are individual elements such as eyes, nose, mouth, and ears, each of which has its own control area, and eyebrows have partial restrictions such as being above the control area of the eye. The adjacency of must be satisfied as a constraint.

Therefore, the mapping from the real world to the perceptual cognitive world is carried out by being interested and driven by the intention, starting from an individual object and expanding into a partial area of the world. In the process, the subject model is geometrically structured from the primitive data level, as shown in FIG. FIG. 3 shows an example of a geometric model which is a formal model of an image.

When the geometric model is set, it is possible to draw an arbitrary still image starting from a point, define how the points in the still image move with time, and draw a moving image. However, the image database cannot be created without constructing the structure of the object on the geometric model. It has been shown by the inventor that the lines in the geometric model are derived from the invariant structure lines as follows.

[0033] Of these, it is the dividing line that has a small topological structure caused by the deformation of the image due to the deformation of the target, and the contour does not lose the characteristics of the single region as long as it represents the outline of the target. On the other hand, in the characteristic line, the phase structure changes even with a relatively small change. This is also related to the fact that the maximum and minimum points are always the intersections of the characteristic lines, but they easily occur and disappear.

From the above, if the target area is extracted by the edge that cuts out the outline of the target, the feature processing or drawing is performed focusing on the dominant area delimited by the dividing line. Furthermore, with regard to moving images, various transformations are expressed centering on the dominant region. As described above, the geometric feature model is defined by the above-described structure line theory from the viewpoint that the feature has some invariance in the environment.

Next, a segment structure as an individual component of the target is required as a feature model. That is, the geometrical dynamic change of the invariant adjacency relations regarding the control area for the individual components expresses the affective characteristics of the object. Considering that the affective features are composed of dynamic changes within individual control regions and between the control regions, the feature model is expressed as shown in FIG.

FIG. 4 is a diagram for explaining the structure of the characteristic model. In FIG. 4, the direction from the geometrical display to the emotional meaning display is the drawing process, and the opposite sense-to-geometric display process is the image feature processing process. With respect to points and lines of geometric display, the meaning of feature points, edge lines, dividing lines, and highlight lines that describe primitive features is described by the restrictions on the features by the structural lines.

As the geometric model, the above is related to the feature about the unevenness on the two-dimensional plane, but the value of the function for a certain attribute value or a set of attribute values is spatially or temporally invariant. That is also regarded as a feature value. For example, the speed of movement of an object as an individual is invariable in a certain time domain, and the change point expresses a feature. Similarly, a boundary between regions in which the change in the luminance value of a certain object is the same is expressed as a feature as an edge or a structure-expanded edge.

The target model shown in FIG. 2 has the above-mentioned properties,
That is, it defines processes driven primarily by constraints. Although humans can understand the target structure expression by the chain of the conceptual structures of FIG. 4, when performing computer processing, the purpose-purpose arithmetic model necessary for tailoring specific execution to the purpose of the computer is used. I need. The combination of the object model and the process model as described above requires a simple conversion process between the target structure and the representation of the execution process. Such model-to-model coupling defines rules for constraints as follows, and an entire system can be effectively operated by an interactive control system for a rule set.

Next, the coupling between models will be described. Process execution processing is performed based on the data model.
As a general rule, data is given by the client, and the content of the constraint is either defined in the model or specified by the client according to the environment. The types are as follows.

(A) AND constraint: Between attribute values of noun objects. There are the following constraint rules between multiple verb objects for a noun object name.

(B) OR constraint (independence): between instructions to a plurality of items belonging to a certain noun object name, and between verb objects starting from a certain noun object name.

(C) Hierarchical Constraint: Due to the relationship between the upper object and the lower object in the processing procedure in the object network, the attribute value set of the upper object at the present time is subjected to the matching operation by the new operation of the lower object. When the attribute value of a noun object is changed by a certain verb object operation, the matching correction is performed on the noun object group higher than that.

(D) Priority constraint: Priority definition between modifiers, priority of role functions in processes, etc. are defined.

Considering the progress of a process as a dialogue process as an interaction, visualization of process operation →
Object network, interactivity with client → Information integration on common platform, role division between client user and expert, efficient notation of collaborative process → control of role function, make the models join together It will be. To that end, data and constraints related to the user are given to the common platform, and the progress is managed in the displayed object network.

FIG. 5 shows an example of the progress of execution processing in this system. Execution processing in the object network is performed on a request-respond basis, but the control rules for inheritance and composition from the process to data at the other end include the priority regarding the process,
It is managed as shown in FIG. As described above, the inter-model coupling is performed based on the constraint rules among various models of data, objects, and processes, which specifically promotes execution management.

As an example of the object network,
The case of the object network for image drawing processing shown in FIG. 29 will be described in more detail. When a Colored point is clicked with a mouse or the like, the luminance of the data window is displayed on the data window as shown in FIG. 6 according to the template of the network. A data window appears for you to specify the value bar, intensity gradient vector, and chromaticity vector.

This fact means that the AND constraint between the attribute values of the noun object is the Color identified as a certain value.
For the object named ed point, it is specified that the data of this noun object should be completed only when the brightness value, the brightness gradient vector, and the chromaticity vector are all completed. Considering this as a Colored point, it is assumed that the point pointing operation corresponding to identify is performed for the points belonging to the Point item.
Move on to ed point processing.

Further, the OR constraint (independence) is that Point may be instructed several times independently, and from Point
It specifies that the attribute value display may be performed multiple times independently under the AND constraint and the identification to the colored point.
The operational specifications for these restrictions are consistent with the general common sense of users, do not require any user operations, and define the user friendliness of an object network as a system.

The hierarchical constraint is that when the user operation reaches the Colored region in the object network, Colore
The template for the d region is shown in FIG. 7 from the template of the specified colored line in each horizontal scanning line, that is, the template of the colored region segment.
As shown in, the coordinate value (X, Y) on the horizontal operation line that the Line focuses on, the brightness value L, and the brightness gradient vector (Lx, L
y), the chromaticity vector (Cx, Cy) is structured into a list for the abscissa X.

At this stage, when a subordinate object is added or deleted by a new operation, the contents of the template shown in FIG. 7 change, and the new or deleted X value changes the structure of the superordinate object, resulting in a new noun. Make consistent adjustments between object hierarchies. For this reason, the hierarchical constraint rule is driven in the system to perform the above matching processing.

In the present invention, the above constraint rules are system constraints incorporated in the system itself of the extended function language Extensible WELL, and for a given object network, a user-friendly operating environment based on the constraint is set. give. That is, the noun object name currently being processed on the object network regulates the system as a state. This change in state specifies the relationship with the constraint rule. This situation is shown in FIG.

FIG. 8 shows the relationship between system constraints and WELL. FIG. 8 (A) shows execution control under run-time system constraints, and FIG. 8 (B) shows interactive control according to constraint rules when designing an object network. Shows an example of. The user inputs instructions and data to the well system 20 via the common platform 30, as shown in FIG. Common platform 30
There are an operation window 61 for displaying an object network, a data window 62 for displaying data, and a message window 63 for displaying a system message. These allow instructions and data intended by the user and the system to be displayed. The execution result is displayed.

The well system 20 stores an object state history 71 for storing a history of object states and the current / operation object 7 currently being operated.
For the number 2, the AND control 7 is executed by the execution control unit 76.
3. The OR constraint 74 and the hierarchy constraint 75 are checked, the execution of the process is controlled based on these constraint checks, and the execution result is recorded by the data recording unit 77.

When designing the object network,
As shown in FIG. 8B, common platform 3
0 displays the object network and the connection structure between the referenced modifier objects (adjectives and adverbs), allowing experts to use constraints based on the models of these systems. The system 20 can define a process model for execution processing by creating a database of modifier priorities or creating a template table of objects based on the design related to the constraint. It is like this.

FIG. 9 shows a flow of processing related to the operation control system according to the runtime system constraint (AND constraint). The AND constraint causes the user to specify a plurality of attribute values of the noun object by the following processing.

(1) The user designates a noun object name by operating on the common platform.

(2) The well system sets a template with the AND mark turned off as a definition preparation process. Then, the AND condition is set and the data window is opened and displayed.

(3) The user performs the attribute value defining operation in the data window on the common platform.

(4) The well system stores the definition data in the corresponding position on the template and displays the contents. Also, the ADN mark in the corresponding template part is set to the on state.

(5) The AND constraint is checked, and when all the attribute values are complete, the next process is performed. If you do not have it yet, wait for data definition.

FIG. 10 shows the flow of processing related to the operation control system according to the runtime system constraint (OR constraint). The OR constraint allows the user to freely select the verb object to be selected next.

(1) As the process progresses, the Well system checks whether or not there are a plurality of verb objects in the executed object, and makes any verb object selectable among the plurality of verb objects.

(2) The user selects the next verb object.

FIG. 11 shows an example of layers for explaining the layer constraint. In the process of progress of the object network, when the lower object is selected or deleted according to the instruction of the lower object while the upper object is already processed, the attribute of the upper object defined in the higher object Value data needs to be designed and coordinated.

In order to realize such a hierarchical constraint, the object network has a hierarchical structure as shown in FIG. 11, and objects A and B defined as local top-level objects are shown by square marks. It is assumed that the status display of is being performed. In such a case, when designing the attribute value of the object F to which the triangle mark is added, the modification of the attribute value of the object F is performed with the contents of the square mark objects A and B without changing the already set template structure. It is necessary to proceed to the design of setting the attribute values of the objects A, B while setting the attribute values of the objects C, D, E.

When a new triangular object H is newly added or deleted, the object name belonging to the object name is added or the specified specific object is deleted. In these cases, it is necessary as a matching operation to prepare for changing the content structure of the template on the path of the object to the upper objects A and B.

Because of the above, the execution control unit of the Well system stores the local highest level executed object name in the object network currently being processed as a state and the current execution object. There is a current / state memory that stores names. FIG. 12 shows the flow of the hierarchical constraint processing. The trace operation at the upper level in FIG. 12 means that the preparation of the definition and the definition operation should be performed in order to set the template content for each object for matching. Therefore, when the setting is not completed in the object network, the message is displayed on the common platform.

Next, the part of speech of the object related to the present invention will be described. In the Extensible WELL extension language, all services are "User puts a point."
It is expressed in a predicate format such as, and is executed, and its use differs depending on the part of speech of the object to be handled. Therefore, a parameter for distinguishing the part of speech for each object is added to the template.
This makes it possible to distinguish whether the object refers to the state of a service, to an operation, or to those constraints.

Since the form of the service is expressed in the predicate format, the contents of the service differ depending on the connection shape between the objects. For example, there are an object (verb + noun phrase) for a verb, an adverb (verb + adverb) for a verb, a noun phrase (adjective + noun phrase) for an adjective, and the like. Priority exists for the connectable objects to the objects corresponding to these. For objects that require a priority order to establish these priorities, the priority order table is referenced to determine the connection order.

FIG. 13 shows an example of the connection order of objects. For example, as shown in Figure 13,
There is an adjective object called featured, these are po
When modifying the int noun object, refer to the precedence table and connect colored to point first
Connect d to the colored point.

Further, due to the layer constraint, the contents of the lower layer have priority over the contents of the upper layer. This means that if some modification or the like occurs on the object network, it will affect the contents of higher layers. As a result, in the object network, the objects existing in the lower hierarchy are more consistently adjusted than the objects existing in the higher hierarchy.

The template and data schema of each object will be described below. Currently, in the extended function language Extensible WELL, there are four parts of speech: noun, verb, adjective, and adverb. Each template shape is shown here. These contents are determined by the expert who sets the object.

[Noun Object] The necessary elements in the template of the noun object are the noun object name, the parameter indicating the part of speech, and the temporal constraint (tem).
It is a pointer for a portrait that indicates a (poral constraint) and an arbitrary template for that object. In addition, there is a priority table that shows the priority order of adjective objects that can be connected to this noun object.
The connection shape is determined by referring to it.

[Verb Object] The verb object is a verb object name, a parameter indicating a part of speech, input data (a noun phrase in a state before execution), output data (a noun phrase in a state after execution) of the object,
A textual statement that indicates the intended use of this verb object (this is for displaying the intended use to the user through the message window), temporal constraint And pointer for constraint indicating model constraint
Composed by.

FIG. 14 shows an example of a verb object template and a data schema.

In the template of FIG. 14, the verb object name is the name of this object. Part of speech I
D is a parameter representing the part-of-speech of this object. For example, the noun is "0", the verb is "1", the adjective is "2", and the adverb is "3". inputdata specifies the object name of input data, and output data specifies the object name of output data. text is a description of how to use this object, constraint is co
Indicates a pointer for nstraint.

The generic verb objects may have different uses even though they have the same name. In this case, depending on the state of the system (a product expression of input data and output data, or constraints such as adverbs), it changes from a generic object to a state object and then to a concrete object to determine the content of the service.
This is determined by the process construction layer and managed by the motion control system.

[Adjective Object] The adjective object is an adjective object name, a parameter indicating a part of speech,
Consists of a pointer for constraint and an arbitrary template showing its contents. Also, the order of connection with the noun object is determined by referring to the corresponding priority table.

[Adverb Object] The adverb object is an adverb object name, a parameter indicating a part of speech, and cons.
It consists of pointer for traint. Also, the connection shape with the verb object is confirmed by referring to the priority table corresponding to this. The above are the system constraints embedded in the WELL, but in addition to these, there are constraints regarding the services specified by the user.

Next, service division will be described.

In order to realize a useful agent function, it is necessary to divide a complicated service into fine parts. The division is roughly classified into two types. Layer division and parallel division.

The layer division is expressed in a nest format and has three states, pre, in, and post. In the pre-state, the necessary pre-processing is performed, in the in-state the actual servicing for the next layer is performed, and in the post-state the post-results made are validated.

Parallel division is performed on services having mutual exclusivity. The divided services can be executed in parallel. The constraint classification is as follows. The divided services are managed by using constraints.
Temporal constraints are used for layer division, and morphological constraints are used for parallel division.

In the temporal constraint, necessary conditions are treated as an AND constraint. In other words, if all the necessary conditions are not met, you cannot move to the next state. Consider a simple service in the object network shown in FIG. 29 in which a certain point is written on the screen. This service is a "draw up Point" on the Extensible WELL extension language.
Is expressed as The object of the noun object "Point" is attached to the verb object "draw up". The coordinate data of the point in this "draw up Point" example can be said to be a temporal constraint in this operation.
An OR constraint (independence) is applied to processes having independence, and parallel execution is possible.

The form constraint is used when integrating the service results issued independently when the above-mentioned parallel division is performed. For example, when synthesizing images written independently of each other, their relative sizes, orientations, angles, etc. correspond to them. In addition, process constraints are included in this and are called hierarchical constraints. When a new operation is performed on a certain lower object, it is necessary to adjust the matching ratio of the upper object accordingly. Furthermore, there is a priority constraint as a constraint on the priority of the role function described above. This is used to determine the highest priority party in the process.

The constraint process is implemented as follows. When performing constraint processing using the constraint conditions as described above, the definition preparation (Defining process
ss) and define operation are defined and managed. The basic concept of preparing the corresponding template in the definition preparation and storing the data in the template in the definition operation is inherited. However, the template itself must be dynamically changeable in order to realize the constraint processing. That is, it is necessary to manage the template group and the template list. Constraints determine the template that is dynamically changed.

FIG. 15 is a diagram for explaining constraint processing and inclusive logic. In the ExtensibleWELL extension language, data must be defined in a value-oriented manner, and its value must already be defined. On the other hand, the constraint is name-oriented, and the value itself is not defined. Only name reference is possible, as shown in FIG. 15, by giving a possible world to the inclusive function, it is extended and the value is defined.
It is equivalent to the data. The dynamic template modification described above is also required for this inclusive function.

The above-mentioned methodology of managing data and constraints in a unified manner is realized by an inclusive logic formula. The data is represented by α (β), and the constraint is represented by α {β}.
This α {β} is a predicate expression of the property of α.
It is described as a set of {j}. Object β has attribute α. {} Represents that an object β that satisfies the attribute α is generated by giving a possible world. This generation is constraint processing, and there are two types of generation methods. Database search and procedure.

As an example of database search, Highligh
There is an extraction of t Point. From the specified points on the horizontal scan line, those having a highlight value above a certain value or the maximum value specified by the expert are extracted. In this case, Highlight Point can be regarded as an adjective because it modifies Noun, which is Point.
Used as a determiner for t.

As an example of the procedure, an example of applying a point symmetrically with respect to a certain object is applicable. In this case, the object Point has the attribute that it is symmetric with respect to an object, and the agent role server calls the procedure required to create a Point that satisfies that condition. The Agent Role Server knows who has the highest priority for this work and will request it. In this case, the case where the user having the highest priority is a human and the case where the machine is a server are considered equivalent except that the interfaces are different. That is, the user is also one of the servers.

The value thus obtained is stored in the template determined as the defining operation, and the process proceeds.
A service constraint is expressed as a relationship between attribute values of an object or between attribute values of multiple objects. For example, the symmetry of two lines requires that the two lines are mirror images of the line of symmetry. In this case, the constraint on symmetry is extracted from the constraint database in the system, or the user defines it as a constraint relational expression using the attribute value of the object. In this case, a request is made to the server that satisfies this relationship. In the case of data, the user is the server as described above. Therefore, in the case of the constraint, the definition preparation and the definition operation are set for the server as well as the data.

This situation regarding the service restriction is the same as that described in FIGS. 14, 15 and 16 of Japanese Patent Laid-Open No. 7-295929. It also relates to the well system support and constraint processing shown in the publication as a purpose-specific parallel execution mechanism of the common platform function by the agent function. As described above, the data model, the object model, and the process model have a hierarchical structure, and the existence of the coupling between these models converts the user's intention into concrete execution by the computer. This conversion process is a goal calculation model, and is specifically expressed as a process model. The user's intent for this process is realized by the Well system by manipulating the object network represented on the common platform.

Therefore, in the process model, the calculation process is controlled by the common platform between the agent role server and the user, and the agent role server (agent r
ole server) is the specific role server (sp
ecific role server) and request the service to the agent role
The structure is returned to the server.

In this structure, for example, as in the hierarchical agent role server shown in FIG. 16, the agent role servers are hierarchically configured, and the upper agent role server is the lower agent role server. -It is also possible to configure the server to be regarded as a specific role server. The above-mentioned dialogue process existing between the servers is realized by the concept disclosed in Japanese Patent Laid-Open No. 7-295929 so as to maintain (a) visualization, (b) dialogue, and (c) cooperation.

In the realization of the inter-model coupling method, when a noun object is designated in the operation of an object displayed on the object network on the user's common platform, the attribute value structure is first used as a model for the data. A template based on is required, and for that, the following processing is performed.

(A) Definition preparation process (defining proces
Call the template from step s) to the work area of the well system and set the data window.

(B) The user specifies the data window set on the common platform and defines the data value. This stage is the define operation. Here, there is a constraint as a property between the attribute values of the noun object, and in order to satisfy the constraint, it is necessary to start the process of obtaining the defined and undefined attribute values. is there. The agent role server running here is the role of the specific role that is responsible for seeking a solution that satisfies the constraints.
Give the server a property that specifies the attribute value set as a parameter and the constraint, and request the solution. Based on the parameters given by the agent role server, the specific role server returns a solution for the constraint as a reply result.

The above is (a) data corresponds to communication between the user and the agent role server, and (b) constraint corresponds to communication between the agent role server and the specific role server. It is considered that the user is a kind of specific role server in the well system, and the user himself gives a solution so as to satisfy the user's intention. Thus, the Well system can proceed based on the same process model for data / constraints.

As an example, an example will be described in which image correction is performed by the continuity constraint. Image continuity guarantee and constraint drive are performed as follows.

[Regarding Image Continuity] When changing the shape of an object existing on an arbitrary image, that portion is designated in the form of a region, and a movement vector is given to each main point forming the region. As a result, the inside of the region can be continuously deformed. However, its characteristics change significantly around it due to deformation. Specifically, due to the movement of the area or the movement of the area, the continuity with the case where the movement does not occur is lost, and a discontinuous portion that clearly indicates that the area has moved appears. In this case, even if the condition of region deformation is satisfied, it is not satisfied in terms of image continuity, and the image becomes unnatural visually.

FIG. 17 shows an example in which a discontinuous portion of an image is generated as a region moves. The solid line is the area before the movement, and the discontinuous portion appears by moving the area as shown by the broken line. In order to guarantee the continuity of an image having such an area, the system side needs to perform processing to prevent these phenomena. Therefore, constraint drive is used.

[Regarding Constraint Driving] Constraint driving refers to a driving method that uses substitute data due to constraints for data deficiencies. For example, when a certain area is moved, if the amount of movement is large, the continuity with the peripheral portion is lost. Instead of letting the user specify the correction area when the continuity is lost, the system side reserves the correction area in advance and executes the processing based on the area. At this point, if the processing result is satisfactory to the user, the process proceeds to the next process. However, the processing result is not always satisfactory to the user. Therefore, reprocessing is performed, or processing is changed such that the user specifies a correction area. This indicates that there is a plurality of processing methods, and even if one processing method cannot continue, switching to another processing method enables the continuation.

The formal model of the object model describes the template structure in the data model as an attribute value structure. This, in some cases as data,
Expressed as a constraint in some cases. As a feature model of an object, there is generally a geometric feature such as a structure line as a certain constraint, and when these segmental structures are expressed by the relationship between regions in the image, they are also related to the kansei model, It is common for the pointing operation to control the system through basic operations like a data model.

The above means that the constraint has a hierarchical structure. For example, when the above-mentioned continuity constraint is given to the relation between dominant regions, the transformation of the dominant region is necessary first, and for that, the feature points that define the dominant region are given, and then the hierarchical structure that defines the region is necessary.

FIG. 18 shows an example of image hierarchical constraint processing.

By the process of FIG. 18, the feature points are changed from the region (Region) to the point Point on the region on the element network of the object network shown in FIG. 29 to obtain a new region, but the original attribute value is used. If the value is given from the value, the continuity becomes inconvenient. Therefore, a correction is made to eliminate this discontinuity. When performing the execution process across such a hierarchy, the constraint process is performed as a system constraint by the Well system, and a process for newly performing image correction is activated.

The above description means that the hierarchical nature of the system matches the hierarchical nature of the model, and the model defines the method of converting from an abstract object to a concrete object, and the reverse conversion is also the same. . In addition, the agent role server interacts with the user's intention by means of an object having an abstraction, and the materialization mediates the agent role server by the parameter, and the specific role server communicates with the user. The system communicates indirectly. But,
The contents of the processing are realized so as to make a specific request to the user.

FIG. 19 shows a flow of an interactive process regarding a continuity constraint which is a kind of service constraint in the moving image drawing system.

(1) When the user intends to draw a moving image, the user performs a request operation for a still image, which is one unit of the moving image, via the common platform.

(2) The agent roll server calls the specific roll server for the database search operation by the requested image name, and the specific roll server extracts the corresponding image name data and executes the search process. To do.

(3) The specific roll server is
The image data corresponding to the requested image name is accumulated and returned to the agent roll server.

(4) The agent role server performs a relay operation and displays the requested unit image via the common platform.

(5) The user performs an operation for requesting that the displayed image be divided into control regions by the concave-convex dividing line or the like.

(6) The agent role server receives a request for extraction of the control area via the common platform and calls the specific role server.

(7) The specific roll server is
Structure lines and constraint equations for feature processing are extracted from the database for the corresponding image, and execution of dividing line processing is requested from the constraints.

(8) The agent role server performs a relay operation and displays the control area data including the feature point display on the corresponding image via the common platform.

(9) The user gives an instruction of the partial control region by the instruction of the set of characteristic points and a deformation instruction of the partial control region by the instruction of the movement amount of the characteristic points. At this time, an instruction via vector display or the like for designating the modification method of the partial area by the common platform is transmitted to the agent role server.

(10) The agent role server requests the specific role server to specify and transform the assigned partial area.

(11) The specific role server is
Upon receiving a request for the modification processing of each partial controlled region, the modification processing is executed. Specifically, based on the motion vector specified as the transformation method for each partial dominant region,
Movement or the like is performed for each pixel in the partial control region.

(12) The agent role server displays the control areas before and after the transformation through the common platform for the integration of the partial control areas.

(13) The user requests the evaluation and application of the continuity constraint condition of the adjacent governing region, and inputs the parameters required for the application.

(14) The common platform conveys the parameter regarding the continuity constraint to the agent role server, and the agent role server uses the specific role for driving the continuity constraint processing of the adjacent area.
Make a roll server call.

(15) The specific roll server is
Correction is performed by the continuity constraint in the adjacent area. The correction amount for each partial controlled area is calculated and transmitted to the agent role server.

(16) The agent roll server integrates the correction amount and displays the correction image via the common platform.

(17) The user checks whether or not the deformed image is acceptable, and approves it if possible. If acceptance is not possible, reprocessing is instructed.

(18) The above processing is repeated to successively draw moving images.

(19) If all the drawn moving images have been authenticated, the process ends. Although there is some overlap with the above description, a method of interactive operation control in a system using the extended function language Extensible WELL will be described next.

In order to realize this system, a system that acts as an interface is required in order to support various types of services and to smoothly and effectively carry out cooperative work between users and systems and between service processes. is there. That is, there is a need for a system that accurately monitors, displays, and controls the operating state of the processing system. Hereinafter, this system is called an interactive motion control system, and its configuration is clarified.

The objects of service of this interactive operation control system are as follows. In this Well system, it is intended to enhance the effect by interactively performing the cooperative processing. Therefore, the target of the service of the motion control system can be roughly divided into two. This is when a user requests a service and when a plurality of specific role servers perform cooperative processing. The control flow of the former operation control system is shown in FIG. 20, and the control flow of the latter operation control system is shown in FIG.

Each operation control includes two constraint processings,
In other words, it can be realized by interactively performing a modal constraint that restricts the work content by a dialog with the user and a temporal constraint that checks the completion of the work state.

First, operation control when a user requests a service will be described.

When a user requests a service, it is necessary to guide the operation in the correct direction in order to carry out the work properly. For example, in the process of creating a moving image, if you accidentally issue a request to move a picture without drawing a still image,
Since there are no pictures to move, you may not know where to make the request. Therefore, it is necessary to limit the request intent to services that can surely execute the user's request.

In this operation control system, the generic data management system and the process construction layer prepare the process for the work based on the service requested by the user. The motion control system can be configured to keep track of the work progress of the current service, so that only the possible processes are given permission to execute. This is the part that performs the morphological constraint processing of the motion control system. The working status of the service can be grasped by setting a flag.

The operation control when checking the service preparation is as follows. As a control required when a user requests a service, it is necessary to check whether the system is ready to execute the service specified by the user. In the Well system, as described above, in the processing process corresponding to each field, the processing form represented by a graph structure in which a state is a noun object (node) and a state transition function is a verb object (branch) is -Define as a network and realize services based on the object network.

When trying to execute each service, activation of this object network is not completed,
If the work cannot be performed, the processing of the branch is not completed, and the work to the next node cannot be performed, it may be impossible to execute the service.Therefore, the operation control system should check and control them. Become. With regard to these controls, a check can be realized by setting a flag when the preparation is completed. This is one of the constraint processes of the motion control system and is a part that performs temporal constraint process.

The operation control when the data at the time of service execution is insufficient is as follows. It is necessary to check whether there are sufficient constraints to execute the service requested by the user. For example, considering a service that collides two objects in the example of moving images, after extracting each object, constraints such as size and motion data of each object are required, and there are no such constraints. Cannot be processed to collide with. If it is found that there are insufficient constraints in event driving when attempting to make a collision, a request is made to the user or service module (set of functions that perform processing corresponding to each field). Of course, the motion control system will not be called if all the data necessary for execution are available.

In general, the constraint condition checking method uses a method in which the verb object inquires of the data management system about the constraint conditions required to execute the service, and the check is performed based on the inquiry. The operation control in the hierarchical constraint is performed as follows.

Based on the service process expressed in the object network of the image drawing system by the extended function language Extensible WELL, the higher-order object is newly operated by the new operation of the lower-order object based on the relationship between the higher-order object and the lower-order object. It is necessary to perform the hierarchical constraint processing in which is corrected. Data (constraints) may become insufficient when modifying the upper-level object, but the operation constraint system recognizes that the data is insufficient and requests the necessary data by interacting with the user or the service module.

FIG. 22 shows an example of control by the operation control system. An example of actual control using the operation control system described above will be described with reference to FIG. This example
This is an example of a case where work is performed in an object network that corrects image discontinuity caused by deformation of an image such as a face image. In the process of requesting the constraints necessary for work,
The party creating the constraint is an example of a user. It is assumed that the motion control system selects a service that changes the facial expression of the facial image after performing motion control that limits the services that the user can perform from the current state as described in FIG.

This object network changes some areas in the face image and judges whether or not the area is within the constraint set so as not to be discontinuous in view of the relationship with the adjacent area. Become. Here, if the change is within the constraints such as a small amount of movement, the following work will be performed. However, if it is found out that it is outside the constraint range, problems such as image discontinuity occur, and it is necessary to correct the dominant region, which is the region generated by the dividing line that separates the shading due to the unevenness of the image. .

Outside the bounds of the constraint, the object network tells the motion control system that the constraint process, or modification, is needed. The motion control system determines a verb object (branch) in the object network that changes the area and issues a work request to the object network that modifies the control area. At this time, as described with reference to FIG. 21, the operation control system performs the operation control for checking whether or not the object network for modifying the control area is ready as the temporal constraint processing.

When it is found that the preparation is completed, the object network for modifying the control area is activated. However, the constraint conditions such as designation of the control area are required. You will have to request. The motion control system will perform the motion control as a morphological constraint means by requesting this constraint by interacting with the user through the manager window on the display and supplying the constraint necessary for execution.

The data to be handled by the operation control system in the role of the interface between the user and the service module are listed below.

Service name requested by the user (may be a concrete name of an object, an object network name, or a role name) Data (constraints) necessary to execute the service requested by the user Object network performing work Name Current node name Parties (user, service module) Service module name for creating constraints Flag to determine if the server is in a runnable state Required to execute the service requested by the user Flag for checking whether or not is sufficient. These data are minimum data necessary for the operation control system to be general-purpose. The following shows how each type of data is used in the motion control system.

First, the service name requested by the user is data for the operation control system to know the request of the user, that is, the service the user is trying to perform. By recognizing the service name requested by the user, the data management system is requested for the data necessary to execute the service. A flag for checking whether or not this data is sufficient as the data required to execute the service requested by the user is used to check whether the service can be executed.

As described above, when it is found as a result of the check that the data is insufficient, the user or the service module supplies or creates the data as a morphological constraint process. The object network name that you are currently working on and the current node name are used together with a flag to determine whether the server is in a state where it can be executed, or when other processing is being performed on the server or processing has been completed. It is used to perform temporal constraint processing that checks for the absence.

The data party is the operator who creates the data, and when the data is insufficient, the data party requests the data and receives the necessary data. The service module name for filling the constraint is data for requesting a service module capable of performing data creation work when the parties have insufficient data in the service module.

As described above, it can be understood that the operation control system is a system which plays a role of an interface for allowing the user and the service module to interact with each other and effectively carry out both tasks. In addition, it corresponds to the characteristics of the extended function language Extensible WELL, and it can be seen that the main control can be realized by two constraint processes, system constraint and service constraint.

The role function and the dialogue function will be described below. The role function and the interaction function described below are also the functions that have already been described directly or indirectly in the above description, and refer to the above description for necessary parts.

FIG. 23 is a diagram showing the definition of roles. As shown in FIG. 23, a role is a unit defined as a structure of an object network and functions as an execution process. A role is given a name (role name), and the name is used to refer to inside and outside the system.

Within a certain role, the relationship between a plurality of object networks is defined as a relational expression between the attribute values of the objects by the constraints defined for the objects of each object network. As a result, the operation inside the role is realized as a constraint process. The role may be composed of one object network.

In the system of the present invention, a plurality of roles perform execution processing, and in order to satisfy the user's intention comprehensively, cooperative operation between roles is required. For that purpose, it is necessary to enhance the interactive function between roles and provide a free communication form. Further, in order to satisfy the user's intention, it is necessary to provide an efficient dialogue function between the user (which can be considered as one of supporting roles) and the system that provides the service. As mentioned above, it is the common platform that is responsible for the interface function between the user and the system. This common platform provides a visual interface between the interworking roles to manage aggregated run state data and to communicate with the role of executing each service module as appropriate. I do. Specifically, an object network is displayed on the common platform, and when you operate on it, a data window for displaying data and constraints is displayed on the common platform at the same time as the object network. It When a user or the like gives a value or a property regarding data or a constraint, the object network plays a given role by executing an execution process.

In the system of the present invention, it is necessary to provide an efficient interactive function between the user and the system and between a plurality of roles. As the interactive function, it is necessary to realize the function having the following configuration.

First, as event driving, the client (user or agent role server) requests to realize a noun object on the common platform. The system that manages the common platform plays the role of a server and responds with the execution result. That is, when a role function issues a request command requesting a service from the partner role function,
In response to this, the role function of the other party executes the service, and responds with the result.

As the data drive, when the value corresponding to the attribute is undefined in the template of the object currently handled as a noun object, the setting of the attribute value is requested. If the verb object has a generic property and it is in the process of determining its parameters, the verb object is treated as a noun. When requesting the setting of an attribute value, a data window is first displayed in response to the situation where the attribute value is undefined. Then, on this data window, the client (user or agent role server) is requested to define the required attribute value.

FIG. 24 is a flow chart showing an example of a well function in order to explain an example of an interactive function based on such even driving and data driving.
It is a figure which shows the movement of the process inside the system 20 (or 21, 22). Also, FIG. 25 corresponds to FIG.
It is a processing flow which shows an example of the interactive function based on ent drive and data drive. 24 and 25, the user (or the agent role server) designates the object of the object network displayed in the operation window 100 (step S10).
1). This is because the user issued a request based on the intention, and is equivalent to event driving.

A template is set corresponding to the user's object instruction (S102). If the target object (objective, etc.) is undefined in the set template, the well system kernel 103 determines that the target object is undefined in the template, and the target object Is requested (S103). This corresponds to data driving.

In the data window 101, the user designates the target object and substitutes this target object in the template (S104). Kernel 10
3 determines whether or not there is an undefined attribute value in the template (S105). If there is an undefined attribute value, the undefined data window is displayed (S106). This corresponds to data driving.

The user defines the undefined data in the undefined data window of the data window 101 (S107) and substitutes the data value in the template (S108). The well system performs execution processing and displays the result in the data window 101 (S109). This is for the request from the user,
This means that the Well System has responded.

By providing the interactive function based on the event drive and the data drive in this way, between the user and the system (between the user and the agent role server)
It is possible to realize a user-friendly and efficient interface on the server, and cooperative operation between role functions among multiple roles (between agent role server and specific role server, etc.). It is possible to realize a communication function that supports the. By implementing the interactive function in the kernel part of the WELL system, it is possible to support various systems, especially software architectures considering personal computer systems.

When performing a collaborative operation between a plurality of roles, the main role which is the main body executing a certain role function,
There is a support role that provides a service function to support the execution of that role, but between the main role and the support role,
It is desirable that the following interactive functions based on common data be provided. The main role executing a certain role function operates under a certain environment related to the main role, and it is necessary to constantly monitor the environmental data related to this environment. However, such a monitoring function can be assigned to a supporting role that supports the main role. The support role shares the environmental data with the main role, and can notify the main role as an interrupt when the environmental data changes. In response to this interrupt, the main role can operate in conformity with the environmental changes.

FIG. 26 is a diagram for explaining a dialogue function between the main role function and the support role function based on the environmental data. In FIG. 26, consider the case where two automobiles are semiautomatically piloted. It is assumed that this system is installed in each car to run on courses that collide with each other. The main role 110 built into one car is
It has an operation method object for performing semi-automatic piloting, and this operation method object is displayed in the operation window 100 on the common platform of the main role 110. Environmental data is displayed in the data window 101 on the common platform of the main role 110. This environmental data is the main role 11
It is transferred from 0 to the support role 111 and shared.

When the environmental data changes and is transferred from the main role 110 to the supporting role 111, the supporting role 111 detects the characteristic property of the environmental data by using this as event driving. The detection of the characteristic property is performed by the characteristic property detection network held by the support role 111. Figure 2
In the case of example 6, for example, the characteristic property that two automobiles come close to each other so that a collision is unavoidable in this state is detected. When the supporting role 111 detects the characteristic property,
The main role 110 is notified of the interruption (responding). When receiving the interruption, the main role 110 sets a motion template corresponding to the operation method object.
If there is an undefined portion in this motion template, and data such as in which direction and how much the vehicle is to be moved is undefined, data drive requests setting of undefined data. The supporting role 111 responds to a setting request for undefined data, detects necessary characteristic properties from the environmental data, and supplies the requested data. When this data is assigned to the motion template, the main role 110 starts an interaction function with the user in order to allow the user to operate the operation method object as an operation guide.

In this way, by sharing the environmental data concerning the environment of the main role between the main role and the support role, and providing the dialogue function based on the event drive and the interrupt between the main role and the support role, It becomes possible for the support role to play a monitoring function for the environmental data of the main role, and the operation of the role function executed mainly by the main role can be supported.

Further, in order to smoothly perform the cooperative operation between a plurality of role functions, the main role function which executes a certain role
It is necessary to enable one-to-many broadcasting for subordinate role functions that have a subordinate relationship to perform the role associated with it. FIG. 27 is a diagram for explaining one-to-many broadcasting from the main role function to the dependent role function.

It is assumed that a control characteristic portion related to a role function related to the main role function appears during execution of a certain main role function while performing a parallel parallel processing between a plurality of role functions. By controlling this control characteristic part in the control attribute value of the control characteristic part on the common platform of these related role functions, the operation of the related role function is controlled by the control characteristic part generated in the main role. Can be done. In this way, the related role function can be positioned as a subordinate role function to the main role. This is the case when the related role is interested in the behavior of the main role, and in the related role, the template of the object that operates depending on the main role contains undefined data.

In FIG. 27, the main role 120 and a plurality of subordinate roles 123 operate cooperatively as a whole system. The main role 120 broadcasts one-to-many to a plurality of subordinate roles 123, and as a data of the same item name as the feature description item specified in the template on its own common platform, the subordinate role 1
Control 23. Therefore, the event from the main role 120
Based on the driving, the support role 121 broadcasts a signal to which the feature constraint data is added. The support role 122, which supports the subordinate role 123, receives the broadcast signal and extracts and confirms the role function name and the characteristic constraint condition of the broadcast source. The subordinate role 123 has a template including an undefined portion, and as an interrupt based on the data drive, the supporting role 12
2. Receive feature constraint data from 2. A template is defined by the received characteristic constraint data, and the plurality of subordinate roles 123 execute the subordinate role functions of the main role 120 in a coordinated manner.

FIG. 28 is a diagram for explaining communication between role functions. As shown in FIG. 28, the role function A, the role function B, and a plurality of role functions not explicitly shown can communicate with each other via a communication environment. A communication support function for supporting communication is provided between each of the role functions A and B and the communication environment. This communication support function has, for example, support roles 121 and 122 in FIG.
Is. Communication between the role function, the communication support function, and the communication environment is performed by an interactive function based on event driving and data driving. The communication support function performs operations such as selection of a communication environment and setting / matching of transmission contents to support the communication operations of the role functions A and B.

For example, in the role function A, the role function B is designated as the partner role function name, and the content value designated by the data item name and the constraint item name is transmitted to the role function B. This controls the content of the execution processing of the role function B.
At this time, each constraint condition of the pre-execution constraint, the execution constraint, and the post-execution constraint is set as the constraint regarding the execution process of the role function B, and the consistency in the cooperative operation is checked. The communication target is not limited to a specific partner among a plurality of role functions, and it is possible to freely select and communicate with the partner.

As described above, according to the present invention,
As a model, a data model, an object model, a role model, and a process model are set, and the coupling between these models is performed by rules centered on constraints, so that the model-driven process defines the execution process. User friendly system use and expert system design can be easily realized.

Further, the data and the constraint are processed by the same processing method in the system, and between the user, the agent role server and the specific role server,
By exchanging the process by exchanging data and constraints, unified management of data and constraints becomes possible.
It will be possible to interact between the user and the service module and to provide an interface that guides them to work effectively.

Also, system constraints are defined by the relationships between objects in the object network,
By controlling the method of constraint processing, it becomes possible to realize a complicated service by a simple method. When the expert designs the object network for service execution, the constraint can be used based on the model of the system, so that the system can be efficiently developed.

Also, by hierarchizing the agent role servers and making the relationship between the upper and lower agent role servers similar to the relationship between the agent role server and the specific role server,
Complex services can be realized with a simple structure, and constrained realization of cooperative processing in the entire system becomes possible.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration example of the present invention.

FIG. 2 is a diagram showing a link between the real world and the emotional target world.

FIG. 3 is a diagram showing an example of a geometric model which is a formal model of an image.

FIG. 4 is a diagram for explaining the structure of a feature model.

FIG. 5 is a diagram showing an example of progress of execution processing.

FIG. 6 is a diagram showing an example of a data window for inputting chromaticity, luminance gradient, and luminance value.

FIG. 7 is a diagram showing an example of a template.

FIG. 8 is a diagram showing a relationship between system constraints and WELL.

FIG. 9 is a diagram showing a flow of processing related to an operation control system according to a runtime system constraint (AND constraint).

FIG. 10 is a diagram showing a flow of processing related to an operation control system according to a runtime system constraint (OR constraint).

FIG. 11 is a diagram showing an example of a hierarchy for explaining a hierarchy constraint.

FIG. 12 is a diagram showing a flow of hierarchical constraint processing.

FIG. 13 is a diagram showing an example of a connection order of objects.

FIG. 14 is a diagram showing an example of a verb object template and a data schema.

FIG. 15 is a diagram illustrating constraint processing and inclusive logic.

FIG. 16 is a diagram showing an example of a hierarchized agent role server.

FIG. 17 is a diagram illustrating an example in which a discontinuous portion of an image is generated as a region moves.

FIG. 18 is a diagram illustrating an example of image hierarchical constraint processing.

FIG. 19 is a diagram showing a flow of an interactive process regarding a continuity constraint which is a kind of service constraint in the moving image drawing system.

FIG. 20 is a diagram showing a control flow of the operation control system.

FIG. 21 is a diagram showing a control flow of the operation control system.

FIG. 22 is a diagram showing an example of control by the operation control system.

FIG. 23 is a diagram showing definition of roles.

FIG. 24 is a diagram showing the movement of processing within the well system for explaining an example of an interactive function based on even driving and data driving.

25 is a diagram showing a flow of processing showing an example of an interactive function based on event driving and data driving, corresponding to FIG. 24;

FIG. 26 is a diagram for explaining an interactive function between a main role function and a support role function based on environmental data.

FIG. 27 is a diagram for describing one-to-many broadcasting from a main role function to a subordinate role function.

FIG. 28 is a diagram for explaining communication between role functions.

FIG. 29 is a diagram showing an example of an object network for image drawing processing.

[Explanation of symbols]

10 users (clients) 20 well system (agent role server) 21,22 well system (specific role server) 30, 31, 32 common platform 40, 41, 42 operation control system 50, 51, 52 Data management system 100 Operation window 101 Data window 103 Kernel 110 Main role 111 Support role 120 Main role 121 Support role 122 Support role 123 Subordinate role

Claims (6)

[Claims] 1. A data processing system that handles data and processing for the data as objects, and processes information by an object network that represents them graphically, when a plurality of processes operate in cooperation with each other. The kernel of the system determines one of an event-driven method and a data-driven method based on an object network that controls the process and state data indicating the state of the process, and the plurality of methods are determined by the determined driving method. The data processing system, wherein the plurality of processes are advanced by a chain of event-driven and data-driven while performing the cooperative operation between the processes. 2. When a user operates the system,
The data processing system according to claim 1, wherein the user is interpreted as one of the processes, and a dialogue between the user and the system is progressed by a chain of the event drive and the data drive. 3. A data processing system that handles data and processing for the data as objects, and processes information by an object network that represents them graphically, each of which is a unit that executes a predetermined function. When a plurality of roles, which are the structures of the network, cooperate to perform execution processing, communication based on data driving and event driving is performed between arbitrary roles of the plurality of roles. Data processing system. 4. The data processing system according to claim 3, wherein each of the plurality of roles is supported by a communication support role that has a function of supporting the communication. 5. The plurality of roles includes a main role that executes a main role and a support role that supports the operation of the main role, and environmental data indicating an environment in which the main role operates,
The data processing system according to claim 3, wherein the main role and the support role are shared. 6. The data processing system according to claim 3, wherein each of the plurality of roles has a one-to-many broadcasting function as a function of performing the communication.