JPH0381828A - Regular base system managing apparatus and method - Google Patents

Abstract

PURPOSE: To present effective data for ineffective discrimination at the program execution by preparing each kind of report by using each kind of data stored in a data base, and selecting defendant rule and a suspicion rule. CONSTITUTION: A monitor program (monitor) 40 is executed at the execution of a production system 10, and each kind of data 42-54 to be stored in a data base are prepared. That is, a pattern matching time (rule 12 right side) 42, operating times 44 of a working memory 18, times 46 of executed condition tests, times 48 of succeeded condition tests, encoding times 50, and executing time (rule 12 right side) 52 or the like are prepared. Then, each kind of report is prepared by an analyzer by using those data sets, dependent rule is prepared first, and then a suspicion rule is selected, and information for supporting the analysis of a rule or data to be changed is prepared. Thus, a user can discriminate the causes of inefficiency at the execution of program, and the executing efficiency of an expert system can be improved, and necessary time can be shortened.

Description

DETAILED DESCRIPTION OF THE INVENTION A. INDUSTRIAL APPLICATION This invention relates generally to computer systems and, more particularly, to tools for efficiency management and expansion used in rule-based expert systems.

B. Traditional Technology Expert systems are computer programs running on general purpose computers that attempt to incorporate the knowledge of experts in a field. Incorporated knowledge can be used by non-experts who, by inputting observable data, can accept hypotheses about the causes of exceptional observations, or who can be consulted in making complex judgments. used by

A typical expert system incorporates data and rules, including facts and relationships. Expert
The database used by a system is often referred to as a knowledge base.

Expert systems use large amounts of CPU resources when executed.

When incorporating expert system technology into a mainstream data processing environment, unless significant efforts are made to improve efficiency, it will compete in terms of efficiency with traditional procedural methods using third-generation programming languages. I can't.

The most popular expert systems are called production systems. The user selects the left side (LH5
) and the right-hand side (RH5). LH of rules
If the three conditions are met, the rule fires and RH5 is executed. RH3 changes the state of working memory. The working memory contains all the factual and data components used by the expert system. When a rule is fired, the rule's LH3 must be rechecked with the modified working memory.

FIG. 1 is a block diagram of an advanced production system IO. The production system IO includes a set of rules 12, each of which has a left side 14 and a right side 16. The production system IO includes:
Also included is a working memory 18 containing facts "known to" the production system 10. A rule interpreter 20 (also referred to as an inference engine) matches the left-hand side 14 of the rule to the working memory 18.

Execute right side 16.

The rule interpreter 20 runs in an infinite loop (referred to as a perception-action cycle). The cognition-action cycle is shown in FIG. 2. The rule interpreter first matches all left-hand sides of the rule to working memory (22);
Matching may be performed by more than one rule. A rule is for its own set of relevant facts, but only one rule can be fired at a time to process any one of the facts. The rule interpreter 20 is
The rules and facts that are fired are selected by conflict resolution. A typical conflict resolution algorithm selects the highest priority rules and the most recent facts for firing purposes. When any one of the rules is selected, the corresponding right-hand side is executed (26), which causes the working
Memory is modified. This cycle is repeated, and all left-hand sides of the rules are stored in the updated working memory 22.
will match again.

The art of writing efficient rule programs has not yet been perfected; general guidelines for creating efficient rule programs are provided by Thomas Cooper and Nanc.
y Rule-Based Programming using Wogrin's rOPs5
sing with OPS5) J (1988) and
Expert system programming with rOPs5 by Lee Brownstan et al.
ming Expert System 1nOPS5
) J (1985), no other publications provide the kind of specialized knowledge available to expert system designers regarding program efficiency.

The guidelines set forth in the documents listed above are guidelines that expert system programmers apply based on their own experience. Such a guideline is based on knowledge of the method covered by the rule interpreter. Current production system rule interpreters are optimized for efficiency, so typically when RH8 of a rule fires, only LH5, which is directly affected by the working memory change, will Verified in verification cycle. This type of rule interpreter limits the matching by using the Rete algorithm. This algorithm generates a number of data structures to store the results of the matching, so if the relevant elements in working memory have not changed, there is no need to rematch. When an element changes, the Rete data structure is examined by the rule interpreter to determine which parts of which rules are affected, and the Rete data structure matches only those rules.

Due to its nature, the Rete algorithm often makes the rule program very inefficient due to a small portion of the left side of rule I11. This is due to the interaction of rules with rules and with data. The reason for this is that a large number of related elements in working memory are screened and compared with each other in various combinations. Screening is done by within-element tests, and comparison is done by between-element tests.

Test specifications are called patterns. Rules and working
The way the memory is structured has a large effect on how long pattern matching takes.

The guidelines that expert system programmers use to improve the efficiency of their programs are broad in scope and not always easy to apply.

Typical examples of such guidelines include numerous working
Avoid conditions that match memory elements, avoid conditions with large cross-products, avoid frequent changes to matched conditions, speed up matching of individual condition elements, limit the size of conflict sets The solution to these problems is to rearrange the conditions on the left side so that the conditions with the most restrictions appear first, and the conditions that match the most frequently changed working memory elements come last. A typical method is to Currently, rule-based expert
Without suitable tools to monitor and evaluate system efficiency, expert system programmers often rely on guesswork and intuition as to where changes will occur.

The efficiency of expert systems varies greatly depending on the data. There are few cases in which the efficiency of a rule program can be assessed simply by examining the rules themselves; when a rule fires, depending on the current state of the system, the amount of data in working memory, and the firing history of previous rules, other Many rules are involved. As such, it is not easy to predict the processing required when pattern matching.

In the field of expert systems, there are no universal rules for creating efficient Rule IIs.

The advantage of the rule program is that most of the data processing is done by the LH.
It is to leave it to 5. LH3 is compact and declarative. In other words, rule LH8 specifies the nature of the data without specifying the mechanism of evaluation. Although writing rule-based applications is simpler than procedural language approaches, it can be significantly less efficient if the program is not optimized. Therefore, the cost of optimizing or tuning a regulation 01-paced program must be commensurate with the productivity gained from writing regulation programs for complex applications. Mechanism is essential.

Therefore, there is a need for a data collection system that is useful for discovering the rules that cause the greatest reduction in efficiency when executing rule programs. It is also desirable that such a system assist the user in examining the efficiency of the application and identifying the causes of inefficiency).

C0 Problems to be Solved by the Invention It is an object of the present invention to provide a monitoring system for a portion of a rule-based expert system that provides data useful in determining inefficiencies in program execution.

Objects of the present invention also include providing a system that represents usable data in a format that can be understood by users interested in improving the efficiency of their applications.

It is an object of the present invention to provide a system for automatically selecting inefficient parts of an expert system application and suggesting a small number of selected candidates to the user for making efficiency-related changes to the application. It also includes providing.

SUMMARY OF THE INVENTION A monitoring system according to the present invention monitors an expert system application while it is being executed. The monitoring system records selected efficiency data related to the operating efficiency of the expert system and writes this data to a database. Select a small number of candidate rules to target and display data to the user to determine what changes are needed to the candidate rules.

E. EXAMPLE The monitoring system described herein is suitable for use with a rules interpreter 20 that uses Rete's matching algorithm.A production system suitable for the present invention is Digital Equipment (:orp,
0PS5, International Busin
Knowledge Too of essMachines
There are l, etc.

Figure 3 shows the procedure for adjusting the efficiency of an expert system application.
System applications are executed (30) and the efficiency of the applications is monitored (32), as described in more detail below. A user interested in the efficiency of the application determines whether the efficiency is satisfactory (34); if so, the procedure ends here. If not, the efficiency data obtained in step 32 is analyzed (36) and the rules within the application are changed (38), and if the way the facts and data are organized in step 38 is also changed, the results of the changes are It can be changed to a more efficient representation in the tested application.

If changed, the application is re-run and the procedure is repeated.

A monitoring program (monitor) 40 in FIG. 4 is executed when the production system IO is executed, and generates various data (42 to 54) to be stored in the database. The data is retrieved as described below and used in analyzing inefficiencies in the production system IO.

Methods of actually monitoring the efficiency of a running program are well known. Once the information to be monitored is determined, it is easy to form the monitor 40 by modifying this method. In efficiency monitoring systems, data selection is generally considered a more important and creative task than writing the computer code that actually performs the monitoring.

The first data set 42 that monitor 40 collects is pattern matching time. During this time, the rule interpreter 20 uses the left side matching (
Pattern matching time data 42, preferably defined as the time to perform steps 22) and conflict resolution (24), is stored in cumulative form for each set II individually. Thus, at the end of execution of the expert system, the contents of the collected data set 42 will be updated for each RH3 of the rule, as a result of the changes made to working memory when the RH3 was fired. This is the total time taken to resolve conflicts.

The contents of the second data set 44 collected are RH5
is the number of working memory operations caused by the execution of . This number is stored for each class of elements in working memory.

And it is desirable to store it every cycle. Allocations, deletions, and updates of working memory elements are tabulated individually. Therefore, at the end of the execution test, the contents of the collected data set 44 are:
The entries for each class affected by the previously fired RH5 are shown for each cognitive-action cycle. The contents of each entry tabulate the total number of assignments, deletions, j5s, and updates performed on members of the class in that cycle.

Third data set 4 collected by monitor 40
6 is the number of times each condition test for LH3 of the rule was logically required to be performed. Each time a condition test is executed as a result of RH5 being fired, the counter for that rule is incremented.

At the end of the execution test, the counter for each condition test is 1.
Indicates the number of times a conditional test was executed.

Monitor 40 also collects the number of successful condition tests 48. A second counter is provided for each conditional test, this counter is incremented for a certain conditional test only when that test is successful.

A record of each pattern is maintained separately. The number of matches is the number of combinations of working memory elements that satisfy the pattern and is obtained by scanning the Rete data structure. It is desirable to keep only the maximum number of matches that occur for a pattern, so the number of combinations of working memory elements that satisfy the pattern is obtained from the Rete data structure each time the pattern is matched;
It is compared with the previous maximum value of the pattern. Only large values are retained.

Data set 52 collected next by monitor 40
is the RHS execution time of the rule.

This is an accumulated value for each rule, so at the end of execution, the time actually taken to execute the right hand side of the rule is applicable to each rule.

Data set 5 last collected by monitor 40
4 is each entry of a cognitive-action cycle executed by the expert system. The content of each entry is the total verification time of the cycle. As mentioned above, this should preferably include conflict resolution time.
Each entry includes the size of the contention set and the ID of the rule that fired to start the cycle. The size of the contention set is the number of combinations of working memory elements associated with all rules that are allowed to fire. It also includes the number of inter-element condition tests and intra-element condition tests executed in the cycle. This information is indicative of the actual tests performed and is based on data set 46.48.
The values collected indicate the number of tests that were theoretically required for each pattern; the actual number of tests is relatively small, since condition tests common to different rules are only executed - degrees; This is because the results will be shared. The data collected by the six monitors 40, whose differences indicate the degree of sharing, is written to a database. Preferably, the ID of each data set is stored in a database.

The storage format within the database is not critical; the collected information may be stored in a text file in the same format as the report described in conjunction with FIG. 5 for ease of handling.

The analyzer of FIG. 5 is connected to a user interface 62. The analyzer of FIG. Analyzer 60 runs on a general purpose computer system and, as described in conjunction with FIG.
Analyzer 60, preferably a program that accesses the database created by monitor 40, extracts data from the database and uses it in several reports (6
4 to 74) to the user interface 62.

The report 64 includes one entry for the right side 76 of each rule, the entries are the number of times the RHS has been fired 78;
and the percentage of CPU time taken for pattern matching after the firing of rule full for RHS, 80, and the CPU time taken to actually execute RHS, and also for all classes affected by the firing of the rule. The number of activities 84 related to members is also included. Activity counts are tabulated separately for the number of class members for which allocations, deallocations, J5s, and updates have occurred. All counts (78-84) are accumulated over the entire time of monitoring.

Report 66 shows class member activities categorized by both reference number 86 (cognitive-action cycle number) and class name 88; in other words, for each cognitive-action cycle, report 66 has a separate entry. , and this entry indicates the cycle number and the number of the class in which at least one member has changed. So, for example, if the members of three types of classes have been changed by - times RHS firings, then In report 66,
Three entries are included for that cycle. The contents of each entry are assigned in the cycle of the class (9
0), the number of working memory elements that have been released (92), i5 and updated (94).

The report 68 includes multiple entries, each containing the number 96 of the match or cycle, the RH5O)ID9B,b that fired and started the cycle, and the left side of the rule that required rematching as a result of the most recent firing. Affected LH5l 00, including ID 100 of
is preferably included in each entry. In that case, if five rules have to be retried by firing the RHS twice, the report 68 will contain five entries, each with one Reference number 96 and ignited R
HS will be the same.

70 contains one entry for each cycle, the contents of each entry are the cycle reference number 102
, the CPU time 104 required for matching and conflict resolution, and the size 106 of the conflict set. It also includes the number of times inter-element tests 108 and intra-element tests 110 were performed.

Furthermore, the RH fired just before the cycle starts
S is also included (112).

Report 72 contains l entries for each rule in the expert system, which is identified by the rule's LH 5114. The contents of each entry are subentries corresponding to each of the patterns included on the left side of the rule. In FIG. 5 there are two pattern sub-entries 116.
118, the exact number of pattern subentries for each LHS 114 of a rule depends on the number of patterns included in each rule.

The contents of each sub-entry are the number of inter-element condition tests required to match LH3 of the rule 120 and the number of successful tests 122. Also included are the number 124 of intra-element condition tests that have become necessary and the number 126 of successful tests. The class name 128 corresponding to each pattern is also included in the corresponding subentry.

Report 74 also includes one entry for the left side of each rule, each entry identifying the LH3l 30 of the rule;
Similar to report 72, it includes sub-entries for each pattern in the rule; in FIG.
However, the number changes as described above. The contents of each sub-entry are the maximum number of inter-element codes 138 and the maximum number of elemental codes 138 that existed for the pattern. These are the values obtained in the collected data set 50 of FIG. Each subentry also includes the number of class members 140 and the class name 142.

As is clear from this, reports 64 to 74 are
Generated directly from data sets 42-54 collected by monitor 40. For example, report 70 is
It is simply the data set 54 in the form of a list. Rule 74 is derived from data set 5o'h). Similarly, data sets 42 to 5 are used for other reports.
It is derived from 4.

Various reports 64 generated by the analyzer 60
74 are defined and displayed according to the principle represented by the cause/effect chain shown in FIG. Figure 6 shows Rule 12
The selection and layout of the collected data shown in the report of FIG. It is desirable to emphasize the nature of the

Figure 6 shows the interaction of events that occur in each cycle of cognition and action, when the right side of the rule is fired (150).
, some of the class members are changed (152), and when some of the class members are changed (152), the left-hand side of a rule is affected (154). As can be seen, only those rules whose left side was affected by the changed class member 152 are rechecked.

Rematching the affected rules 154 causes the condition tests within those rules to be reevaluated (156), which may result in some pattern matches (15).
8), once the patterns in the affected rule III are satisfied, those rules and matching working memory elements are added to or removed from the conflict set (160); The conflict set 160 thus formed by the test conditions and match patterns consumes CPU time while the system program is running.

Execution of expert system applications
Analyzing according to the concept shown in Figure 6 means:
When the right-hand side of a rule fires, the rule interpreter 20 attempts to improve efficiency by paying attention to the rule [1] that caused it to consume excessive CPU time in the next match cycle. A rule that results in such a time-consuming match can be called a defendant rule.

Box 154 indicates that due to the firing of the defendant rule, the LH3 of a certain rule must be rechecked; such rule checking actually consumes significant CPU time. Therefore, these rules can be called suspect rules, and it is necessary to focus on them when analyzing efficiency information.

Next, FIG. 7 shows the operation of the analyzer 60. When operation of analyzer 60 begins, the first step is to select defendant rule 170. If necessary, several defendant rules can be selected and tracked simultaneously, but in FIG. 7, only one defendant rule is selected at a time. The selection of the defendant rule (170) is performed based on the information in the report 64. The defendant rule is preferably the rule that requires the longest CPU time 80 for verification.

The next step is to determine which classes were changed by the firing of the defendant rule (172). Analyzer 60 determines from report 70 which cognitive-behavioral cycles (or matching It can be determined whether a fire has occurred based on the number). This list of verification numbers is available in report 66.
is used to select only those entries with the corresponding reference number. As a result, a report is selected from the entries in the report 66, and from this report, the class names 88 of only the classes changed by the firing of the defendant rule are obtained.

After examining the selection result in step 172, the user can move on to selecting a suspect rule (174). The suspect rule is a rule whose left side was affected by the firing of the defendant rule. This information is stored in the report 68. RH3 ignited from
The information 98 is obtained by extracting only the entries that match the defendant's rules. The name of the suspect rule is used as a filter for other reports for the user (176
- 182), the list of suspect rules is used to select only those entries corresponding to the suspect rules from the various reports.

Once a suspect rule is selected, several reports 176-182 are presented to the user. Based on these, users
Identify the causes of execution inefficiencies. The presentation order of the reports can be arbitrary, but it is preferable that the user selects the order he or she wants.Usually, if there is a problem, 1.
Some reports provide useful information, while others do not.

The user selects the Inference Cycle Activity - Report 1
Uke takes 76. This report includes only inference cycles that include Suspect II. This is obtained from report 70. The user also receives a pattern-by-pattern test and match report 178 for the suspect rules. This is obtained from report 72, in which each pattern of suspect rules is all presented. A pattern-specific matching report 180 regarding the suspect IQ is extracted from the report 74 .

The size of the contention set per cycle is also reported (1
82), which is extracted from the entries in report 70.

In many cases, users will determine, based on their knowledge and experience, that they do not need to see all of the reports just described. Some exceptions may become apparent early in the process, and the user may fix them without checking. For example, if many LH5s are found to be affected in step 174, the user may change the class members or combine several similar rules into a single rule and remove the differences on the right-hand side of the combined rule. may be resolved. If there are any changes at this time, the user will exit analyzer 60 and perform another run test on the expert system.

The types of changes that can be made to the expert system when the above-mentioned reports are checked will be obvious to those skilled in the art and will not be described in detail here, but a brief example of the use of each report will be described below.

The inference cycle-activity report obtained in step 17G can be used to diagnose problems caused by large conflict sets or to locate exceptions to rule firing. Such diagnosis and confirmation is performed by searching for verification cycles that deviate significantly from the standard. The report obtained in step 178 can evaluate the trade-off between the number of matches in the LH5 and procedural search, or indicate the need to reorder patterns in the LH5 to reduce redundant cross-element testing. It can be used to The report of step 180 can be used to diagnose the magnitude of cross-product effects and to alert you to possible dynamic memory spikes.

Retrospective 111186, 188 indicates that the user can also reverse the procedure of the analyzer 60 to examine data regarding the efficiency of additional rules, e.g. , step 1
From 72, a return can be made to step 170. this is,
It amounts to selecting a new Defendant Rule, in which case the Defendant Rule is the rule that also caused the change to be made to the same class that was changed by the original Defendant Rule.

Similarly, if step 174 returns to step 172,
You can create a new modified class.

This corresponds to selecting all the classes that caused changes to the original set of suspect rules and the rules that were actually changed by the original suspect rules. A new suspect rule is now selected, which includes all suspect rules also modified by this expanded set of classes.6 Next, in each report produced in steps 176-182. , all entries corresponding to the thus expanded series of suspect layers NIJ are selected.

As will be apparent to those skilled in the art, the system described above is useful in extracting relevant data while the rule-based expert system is running and determining which rules and data should be changed to increase the efficiency of the system. , provides a monitoring mechanism to assist users in data analysis. This mechanism is based on the efficiency data analysis (3) explained in conjunction with Figure 3.
6) to significantly increase the efficiency of and significantly reduce the time required to increase the execution efficiency of the expert system.

Although the invention has been described with particular reference to illustrative embodiments, those skilled in the art will recognize that various changes may be made in form and detail without departing from the spirit and scope of the invention.

[Brief explanation of drawings]

FIG. 1 is a block diagram of the production system. FIG. 2 is a flow diagram illustrating the cognition-action cycle. FIG. 3 is a flow diagram illustrating the adjustment process applied to the expert system. FIG. 4 is a block diagram of a monitoring system illustrating how data is collected during execution of an expert system program. FIG. 5 is a block diagram illustrating how the collected data is used by the analyzer. FIG. 6 is a diagram showing the relationship between cause and effect of each part of the expert system during execution. FIG. 7 is a flow chart showing the operation of the analyzer of FIG.

Claims (11)

[Claims] (1) A system for monitoring the efficiency of a rule-based application having rules and a working memory containing multiple elements, each rule having a left side and a right side, the match required for execution of the right side of the rule. a timer for accumulating time; a first counter for counting working memory operations performed on each class of elements in said working memory; a second counter for counting the success of the conditional test; a third counter for counting the combinations of the working memory elements that satisfy the pattern each time the pattern is matched; and means for storing information in a database. (2) The system according to claim 1, further comprising a second counter that accumulates the execution time of the right side of the rule. 3. The system of claim 1, including means for generating a report indicating said information stored in said database. (4) The system of claim 1, comprising means for selecting a suspect rule for testing purposes and means for displaying information extracted from a database related to the suspect rule. Management device. (5) The system according to claim 4, wherein the selection means is a comparator whose right side determines the defendant rule that caused the matching cycle time to become relatively long after execution; a first selector that selects the working memory class affected by the execution of the right-hand side of the rule; and a second selector that selects the rule whose left-hand side was affected by the change in the selected class. and wherein the selected rule is a suspect rule. (6) A method for providing information regarding the causes of inefficiency in a rule-based expert system, comprising: accessing efficiency data generated during operation of the expert system; and selecting a set of suspect rules. and displaying efficiency data for the suspected rules. (7) The method of claim 6, wherein the selecting step includes: processing the accessed efficiency data to determine which defendant rules have resulted in a relatively longer matching cycle time due to execution of the right-hand side. and selecting a working memory class affected by the execution of the right-hand side of the defendant rule; and selecting the rule whose left-hand side was affected by the change in the selected class as the suspect rule. Rules-based expert system management method. (8) A method for monitoring the efficiency of a rule-based application, comprising: accumulating the matching time required to execute the right-hand side of a rule; counting memory operations; counting executions of conditional tests on the right-hand side of the rule; counting successes of the conditional tests; and determining working memory elements that satisfy the pattern when the pattern is matched. A method for managing a rules-based application, comprising: counting; and storing information obtained from the accumulation and counting steps in a database. (9) A method according to claim 8, comprising the step of accumulating the execution time of the right side of the rule. 10. The method of claim 8, comprising: accessing a database; selecting a set of suspect rules; and displaying efficiency data for the suspect rules. Method. (11) The method of claim 10, wherein the selecting step comprises: processing the accessed efficiency data to determine the defendant rule whose match cycle time has been relatively longer due to execution of the right-hand side. and selecting a working memory class affected by the execution of the right-hand side of the defendant rule; and selecting the rule whose left-hand side was affected by the change in the selected class as the suspect rule. Rules-based application management method.