NL1037190A - CLASSIFICATION MECHANISM USING A DYNAMIC QUESTIONNAIRE. - Google Patents

Description

Classification mechanism using a dynamic questionnaire.

The mechanism for assigning a category (class) to a person who answers questions in a questionnaire (participant) is called a classification questionnaire. Examples of this are medical diagnoses, where by answering questions about symptoms the most appropriate condition is searched for, or a voting pointer, in which answering questions about political preference seeks a most suitable political party.

A characteristic of classification questionnaires is that the list of questions is put in a predetermined order.

The disadvantage of this is that not every question asked has to be equally relevant to a participant. An example is the first time the questions "Do you own a car?" and "What is the travel distance between you and your work?" and the participant answers this with 'no' and 'more than 10 km' respectively, the question 'Do you use public transport' becomes less relevant because the chance that this is answered with yes is greater.

An existing solution to this problem is by applying logic and by skipping questions in a questionnaire (skip-pattern questionnaires). Here rules are used whereby one or more questions can be skipped at the moment a given answer is given. Examples of this are "Do you have children?" with the answer 'no' being stated, 'Go to question ..'.

With the use of computing power in a computer system, it is possible to go beyond this binary logic, and dynamically determine the relevance of a question in a probabilistic way. Instead of determining the relevance binary with logic, in the model presented here, the relevance of the remaining questions in the questionnaire is calculated after each question and the best follow-up question is asked to the participant based on this relevance. In this way a computer system can be developed where questions can be asked to a participant using a trivial interface, with which the computer system can advise the participant or a domain expert with regard to the outcome candidates.

By using an optimal question sequence, after asking a relevant part of the questions, you can already stop asking questions with minimal loss of accuracy of the classification, or by allowing the system to have more questions than will be available asked, more relevant questions are asked, resulting in a better classification.

These two benefits can also be combined, resulting in a short, precise questionnaire.

The way to determine the relevance of a question is done on the one hand by looking at which questions will make a good distinction between the relevant candidates (as it were, maximizing the entropy of a question), and on the other by looking at how much a question add answered question to non-overlapping information in the classification (with the aim of achieving high objectivity).

How well questions meet these criteria is calculated automatically by modeling both the questions and the influence that the answers of those questions have on the various possible outcomes in a universal way. With the invention, in combination with a random classification model (for example probability, bayesian logic, PCA and the like), the above criteria can be determined.

The invention can be placed in an automated questionnaire, which can be modeled with the aid of (figure 1) in which it is shown schematically how the processing of an automated questionnaire takes place. First of all, a question is selected using the invention (step 1 in Figure 1). This question is then asked to the participant (step 2). It is then decided (step 3) whether more questions can be asked and whether this should be done. If this is the case, the process repeats itself from step 1. The moment there are no more questions or it is decided not to proceed, a classification is performed using the classification inference machine described below (step 4). The result of the classification is then presented in a form that is appropriate for the context in which the questionnaire is held. (step 5)

The operation of the invention uses a random classification model that can be modeled as shown schematically in (Figure 2). Herein, block A represents the aforementioned classification inference machine. It has input h, representing a set with one or more question-answer tuples in which each tuple in the set is represented by a tuple <v, a> in which v represents a question, and a represents an answer to question v. The second entry (e) of the classification information machine represents a set of outcome candidates. Using these two inputs, the inference machine can generate an output set S of the same length as e, in which each element in 5 indicates the relevance of an element from e, given that a participant has answered the questions in the question-answer tuples h with the corresponding answers.

The demand selection from step 1 of (figure 1) is effected with the aid of the invention. The invention can be modeled with the aid of (figure 3) in which an 'Information measure' (block A) and an 'Objectivity measure' (block B) both yield a score (5j and S2) which can be combined by combining these (block S in figure 3) ) make it possible to select a follow-up question (v).

The Information measure (block A in figure 3) is a measure that calculates the amount of new information that a demand offers, and is modeled with the help of (figure 4). First of all, using the answers already given h, and the outcome candidates e an 'Important Set' is calculated (block IS in figure 4). This process described below selects the outcome candidates that are most relevant at that moment (e 'in Figure 4). Score Si is then calculated for each candidate question (v) from the set of candidate questions V and e '(Ent in Figure 4). This process starts with calculating for each possible answer (a) of a candidate question v what the set of scores S is for the candidate outcomes in e 'if a would be added to the current h history. This is done by using the previously described classification inference machine modeled in (Figure 2), in which in this case h, h from (Figure 4) with the tuple <v, a> added, and e, e 'out ( figure 4).

Subsequently, the resulting scores for each outcome candidate are compared and a new set M is made (M in Figure 4), in which each element for an answer of v represents the number of outcome candidates from e 'that has the highest relevance score for that answer. This set therefore shows the expected distribution of outcome candidates given the questions. By then calculating the entropy over this set as defined by Shannon in literature, a measure of uncertainty about the outcome is found (S ^). The higher this number, the more the question contributes to the classification.

The calculation of the 'Important Set' (Block IS in Figure 4) is done by having the classification inference machine calculate the relevance score S for each outcome candidate, given the current h history. By comparing the scores of all outcome candidates with each other, a selection can then be made of the outcome candidates in which a certain percentage g with the most relevant scores is selected. The percentage g is automatically selected with the help of the number of questions that have already been answered. The more questions are answered, the lower the percentage must be, in order to make the 'Important set' increasingly smaller and more specific.

The Objectivity Measure of a Potential Question v (Block B in Figure 3) is a measure that determines the extent to which a potential question questions unanswered factors, and is modeled using the model in (Figure 5). Objectivity is determined with the help of the relevance scores s in figure 5, which is first determined for the outcome candidates (e) and the current history (h) with the help of the classification inference machine (Block A in figure 5). A score s is also determined in Figure 5 for each answer (a) to the potential question v by using the elemental influences that that answer (a) has on the outcome candidates and using it as a history of a question that can then be applied to the classification machine (Block B in Figure 5). The now calculated scores s and s' are the scores of the history and the answers to the question, and are then compared using the well-known Pearson's correlation measure (Block C in Figure 5). This results in a score for each answer. This score is combined into one score S2 that can then be used in the question selection model (Figure 3).

Claims (5)

1. The invention arranges a questionnaire in such a way that more relevant questions are asked earlier in order to speed up a classification questionnaire and / or to qualitatively improve the classification. 2. By means of information that can be obtained from the answers that a participant gives to questions, a strategic change in order of questions is achieved in order to achieve the aim of the first conclusion. 3. By modeling outcome candidates and their elemental influences on the outcome candidates by means of a random classification model that is compatible with the model in (Figure 2) in which a question / answer history (ft) and outcome candidates [e) have a relevance score ( 5) for each outcome candidate is calculated, insight is provided into similarity between already answered questions and potential questions, and insight is given into the amount of relevant information that answers and questions can offer in order to determine the strategic question sequence from claim 2. 4. By using the absolute value of the correlation between the answers already given and the elemental influences that a potential answer has, combined with the chances that a potential answer of a potential question is given as an answer, the similarity between a potential question and possibly previously asked questions determined in order to use it in determining the strategic order of questions from claims 2 and 3. 5. By using the classification model described in claim 3 and the well-known entropy measure, the amount of relevant information that a potential question, given the demand history (ft) and the outcome candidates (e), could yield, is calculated for a subset of outcome candidates who are characterized by their high relevance score at that time, in order to be able to use it in determining the strategic order of questions described in claim 3.