GB2499827A

GB2499827A - Assessing a person's ability to achieve a pre-determined outcome

Info

Publication number: GB2499827A
Application number: GB1203615.8A
Authority: GB
Inventors: Ian Smythe; Amanda Kirby
Original assignee: DO IT SOLUTIONS Ltd
Current assignee: DO IT SOLUTIONS Ltd
Priority date: 2012-03-01
Filing date: 2012-03-01
Publication date: 2013-09-04
Also published as: GB201203615D0; WO2013128174A1

Abstract

A person profile is constructed, comprising a plurality of attributes relating to the person. At least one attribute relates to a disadvantage which adversely affects the person's ability to achieve the outcome, such as disability. A target (outcome) profile is also constructed, comprising a plurality of attributes which are considered relevant to a job. A score for each attribute in the person and/or target employment profile is received and stored. The person's ability to achieve the outcome is assessed based upon a comparison of the attributes in the person and target profiles. The method and system may comprise the generation of a strategy for upgrading any attribute score in the person profile which is less than the corresponding attribute score in the outcome profile. A plurality of models may be constructed using a variety of artificial intelligence technicques so that results can be compared.

Description

-1-

Assessment Method and Corresponding System

The present invention relates generally to the field of skills assessment. In particular, it relates to the field of assessing an individual's ability to achieve a particular goal. The 5 invention lends itself to numerous implementations and applications, and its use is beneficial in a variety of fields such as personnel selection for employment, educational courses and training. Beneficially, the system can also function as a recommendation facility for identifying any deficiencies in an individual's skill set in respect of a pre-determined goal, and recommending at least one strategy for addressing the deficiencies.

10

In preferred embodiments, the present invention relates to the field of education, employment and assistance for those with a disability. More specifically, it relates to predicting intervention and support efficacy and determining any upgrade of skills ('up-skilling') required for any education and employment related situation, irrespective of the 15 characteristics of the individual and their situation.

It also relates to the field of performing educational, employment and disability-related assessment, and estimating and assessing the strengths, abilities, weaknesses and characteristics of the individual.

20

Despite the need and importance for matching of individuals' skills to education and employment, known procedures for assessing a diversity of an individuals' capabilities are not widely available or appropriate for certain sectors of the community or population. The choice of education, training, skills upgrading and employment is currently based on 25 personal preference, motivation and limited knowledge including educational attainment, and general skills, and is restricted by the limited knowledge of the advisor.

However, it is not uncommon for recommendations to be inappropriate due to hidden disabilities or disadvantages. Such factors might include dyslexia, developmental co-30 ordination disorder, ADHD, Asperger's Syndrome, as well as conditions or factors relating to mental and physical wellbeing. As a consequence, the target outcome (education or employment) is not attainable or not sustainable. Only with a fuller understanding of the

-2-

individual, their characteristics and their potential will it be possible to maximize the potential for recommendations which are realistically achievable.

These recommendations need to take into account not only the profile of the individual, but 5 also the profile of the target outcome (e.g. employment) and training (e.g. a course for builders). However, while the individual may appear at one level to fulfill the needs of the outcome or up-skill resource, hidden factors (e.g. reading difficulties or mental health or back pain etc.) may adversely affect the potential to fulfill the course or role.

10 Current appraisal systems involve input from multiple experts, using inconsistent and often contextually invalid systems. It often takes months to provide an appropriate picture of the individual. The sensitivity to the needs of the individual is often questionable. That is, the current assessment techniques may be valid with one individual, but not be applicable to a wider population.

15

The process for assessing an effective intervention (training) is poorly defined at best. In short, the basic procedure for prescribing an up-skilling strategy is on a trial and error basis. Recommendations are made repeatedly until one is found that is effective.

20 Furthermore, an individual must often wait lengthy periods of time before seeing the "experts" who are charged with identifying target outcomes and recommended interventions. A lack of training and integration of services provided by these "experts" and access to appropriate resources, as well as a time and fiscal imperative means that the understanding of the individual and their needs is often superficial.

25

As a consequence, those supporting these individuals, (including teachers, careers offices, disability officers, employment support workers and human resources personnel) have partial knowledge of aspects of the individual. However, nobody has sufficient knowledge to make appropriate, informed choices based on all the information that could be available, 30 Furthermore, there is no mechanism whereby all those involved could contribute towards a better understanding of their needs.

-3-

As a consequence, persons having disadvantageous characteristics (such as mental or physical disabilities, social disadvantages, cognitive issues and other challenges) do not sustain jobs or complete training/educational courses. Unfortunately, this affects those who most need this kind of support.

5

The first step in determining efficient intervention is establishing a correct assessment of the individual's profile, including strengths, weaknesses, abilities and difficulties, as well as motivations and aspirations. This can be a more difficult and technically complex task than it might seem.

10

An example of a known assessment method/system is disclosed in US Patent 7,606,778 (PreYisor). The PreVisor system tests potential employment personnel by posing questions to a job applicant located at an applicant terminal. The person's answered are 'scored' and then entered into the system for the calculation of a ranked table wherein each person is 15 ranked against each other in order from first to last or other comparative ranking. However, the questions posed to the individual are designed to test them in relation to competencies which are shown to be relevant to successful performance of each job type. Such competencies include 'dependability', 'agreeableness', 'leadership' etc. So the system assesses the individual in respect of 'positive' attributes i.e. characteristics which are 20 deemed to be strengths. While an individual may score poorly for a positive attribute, the system does not explicitly test for, identify, evaluate or account for any 'negative' factors or disadvantages which he may possess and which may be prejudicial to the his ability to perform the desired role e.g. dyslexia/reading difficulties.

25 It is also known from the prior art that techniques developed within the field of Artificial Intelligence (AI) may be useful for improving the performance of computer-implemented recruitment systems. For example, 'A Fuzzy Expert System (FES) Tool for On-line Personnel recruitments', Daramola J.O., Oladipupo O.O., Musa A.G. (2010) 'A fuzzy expert system (FES) tool for online personnel recruitments' International Journal of 30 Business Information Systems Vol. 6, No.4 pp. 444 - 462 describes the use of fuzzy logic for such an explanation. Other AI techniques, such as Neural Networks, are also known.

-4-

Thus, there is a need for an effective and reliable way to improve the accuracy and efficiency of the prediction of a person's needs and results, in terms of both up-grading of skills and desired goals.

5 There is also a need for a solution which uses multi-source analysis to identify skills,

abilities and characteristics for an individual, compare them to identified needs and desirable outcomes, and provide evidence-based suggestions of how to achieve those goals. Furthermore, it may be dynamic and can monitor change in the individual, the job, the training and the match between them.

10

Such an improved solution has now been devised.

Thus, in accordance with a first aspect of the present invention there is provided a computer-implemented method for assessing an person's ability to achieve a pre-determined outcome, 15 the method comprising the steps of:

constructing a person profile comprising a plurality of attributes relating to the person, wherein at least one attribute relates to a disadvantage which adversely affects the person's ability to achieve the outcome;

constructing an outcome profile comprising a plurality of attributes which are 20 considered relevant to achievement of the outcome; and assessing the person's ability to achieve the outcome based upon a comparison of the attributes in the person and outcome profiles.

Beneficially, the method may comprise the step of receiving a score for each attribute in the 25 person and/or outcome profile and storing the received scores in association with their respective attributes.

Beneficially, the method may comprise the step of comparing the attribute scores in the person profile with the attribute scores in the target profile.

30

The score in the outcome profile may represent a predetermined threshold.

-5-

Beneficially, the method comprises the step of computing a strategy for upgrading the identified score to at least a threshold level associated with a corresponding score in the outcome profile.

5

Beneficially, the profile attributes are weighted.

Beneficially, the target profile may include at least one attribute relating to a disadvantage which adversely affects the person's ability to achieve the outcome

10

Beneficially, at least one disadvantage may relate to a physical, behavioural, mental health, learning disability, cognitive disorder and/or psychological disorder or any other disadvantage exhibited by the individual.

15 Beneficially, the person profile and/or the target profile may include at least one attribute relating to social, domestic or contextual factors.

Beneficially, for each attribute in the person profile there may be a corresponding attribute in the outcome profile such that the attributes relate to the same features.

20

Beneficially, the method further comprises the step of storing the attributes and/or scores in a form of computer-readable memory.

Beneficially, at least one attribute in the person and/or target profile relates to a skill, 25 qualification or educational or vocational feature.

Beneficially, the method further comprises the steps of:

employing a variety of techniques, such as neural networking, Hidden Markov Models, Bayesian rules, fuzzy logic, classifier systems, Adaptive Resonance Theory, Natural 30 Language Programming or any other artificial intelligence method to construct multiple models of the person and/or target profiles; and comparing the results provided by the plurality of techniques.

-6-

In accordance with a second aspect of the invention there is provided a computer-implemented method for assessing a person's ability to achieve a pre-determined outcome, the method comprising the steps of:

5 constructing a person profile comprising a plurality of attributes relating to the person, each attribute being associated with an attribute score;

constructing an outcome profile of the outcome comprising a plurality of attributes considered relevant to achieving the outcome, each attribute being associated with an attribute score;

10 comparing the attribute scores in the respective profiles; and generating a strategy for upgrading any attribute score in the person profile is less than the corresponding attribute score in the outcome profile.

Also in accordance with the invention there is provided a computer-implemented system 15 comprising one or more components arranged and configured to execute the method of any preceding claim.

A preferred embodiment of the invention will now be provided by way of example only, and with reference to the accompanying Figures in which:

20

Figure 1 illustrates some of the characteristics or attributes relevant to the assessment of an individual, and the sources from which data relating to those characteristics may be gleaned.

Figure 2 illustrates some of the attributes which might be considered relevant for achieving 25 an employment-related target outcome.

Figure 3 illustrates some of the attributes which might be considered relevant for achieving an education-related target outcome.

30 Figure 4 shows an overview of the functionality of an illustrative embodiment of the invention.

-7-

Figure 5 illustrates some of the attributes which might be considered relevant to the profile of a training up-skilling resource.

Figure 6 shows one possible method of implementation using neural network principles with 5 forward and back propagation methods of artificial intelligence.

Figure 7 shows the use of a neural network in the context of a job application.

Figure 8 illustrates how the value of the weighting will adjust according to the distance from 10 the estimate in respect of the group average.

Figure 8 shows a flow of sequences in a typical embodiment using a neural network in profiling a job.

15 Figure 9 shows some example data points relating to different types of attributes.

Figure 10 shows the hardware components which may make up an implementation of an embodiment of the invention.

20 Figure 11 shows an overview of the structure of an implementation in accordance with the present invention.

The present invention comprises a computer-implemented method and corresponding system which enables an assessment to be made of an individual in respect of a particular 25 task or goal. The assessment is a judgement of the likelihood that the individual will be able to achieve the given task or goal. In other words, the assessment process provides an informed opinion regarding the individual's suitability for the task or goal. As the result is an opinion, there is no right or wrong 'answer' provided by the process. The result of the process may include one or more recommendations. This is discussed in more detail below.

30

A user of the invention may be an individual seeking to achieve a goal and desiring an assessment of the likelihood of success, or the user could be another party seeking an

-8-

assessment of the individual's likely success in respect of the goal. For example, the system could be used by an individual seeking to secure a job, or by an employer seeking to fill a vacancy. However, it should be noted that the invention is not limited to applications of employment and/or recruitment, as will become apparent in the following discussion.

5

The assessment process requires a profile to be built of the individual. This is referred to hereinafter as the 'person profile'. The person profile is a set of attributes (i.e. data items) which relate to different personal characteristics. Examples of typical 'person attributes' are shown in figure 1. An attribute can be a characteristic, feature, condition or fact relating to 10 that person.

For example, a profile for a person might include a 'fine motor skills' attribute, or 'literacy level', or 'possession of a driving licence'. The nature, type and number of the attributes included in a person profile is pre-determined by one or more experts, possibly in 15 collaboration with other interested parties (such as an employer looking for a person to fill an employment vacancy). The expert may have expertise in one or more fields such as psychology, education, psychiatry, careers and employment.

The actual data values for the person profile attributes are provided from a variety of 20 sources. This is explained in greater detail below. In certain embodiments, the person profile is designed in respect of a particular target outcome i.e. the attributes in the person profile may be selected according to the needs of a given goal for which the person is to be assessed. Using his expertise, the expert determines which attributes are relevant when assessing the suitability of a person for a given target outcome.

25

Each attribute in the person profile is associated with at least one value (or 'score'). The score is indicative of the person's strength in respect of the associated attribute. The score may be a numeric value, an alphanumeric value, a binary value or some other type, The nature or type of data stored in the person profile is not intended to be limited in this regard.

30

A novel and inventive aspect of the present invention is that the person profile includes attributes relating to disabilities, disadvantages or 'challenges' which may be possessed by

-9-

the person. These factors could include mental or physical disabilities, social disadvantages, cognitive issues and other challenges. In essence, they include factors which relate to difficulties which the person must overcome in order to succeed at the target outcome or which may influence the choice of target outcome and recommended upskilling. In terms of 5 goal achievement, they may be deemed 'negative' attributes as they represent challenges which the person must resolve, mitigate or compensate for if he is to achieve the target outcome. This contrasts with the 'competencies' of, for example, the PreVisor system which forms an assessment using only personality traits.

10 By way of example, the profile may include a score associated with a learning difficulty such as dyslexia, or dyspraxia. Alternatively or additionally, physical and sensory disabilities may be included in the profile. These factors all have an impact upon the person's likelihood for achieving certain target outcomes. However, the known assessment systems do not incorporate or consider this type of information and thus their understanding

15 of the individual is limited, and the results produced by such known systems are neither reliable nor as useful to the user as those provided by the present invention. The system also may have the capability to track changes in the individual, job and up-skilling profiles, and have the ability to re-evaluate matches as changes occur. Thus, by taking a richer variety of attributes into account, the invention is able to provide a more meaningful assessment.

20

Some of the disabilities included in the person profile might relate to 'hidden'

disadvantages, such as learning disabilities, dyspraxia, executive functioning difficulties and so on. Sometimes, the person being profiled may not even be aware that they have a weakness in this area, and so the simple question-answer approach of known systems would

25 not be able to discern data relating to this type of factor. By taking in data from a variety of sources, and using experts to devise the techniques for obtaining the data, the present invention is able to utilise a more diverse range of information and thus build a more comprehensive and/or relevant model of the individual.

30 Additionally or alternatively, the system can incorporate contextual information relating to the person. For example, marital status, location of residency, number of dependent children etc. As above, consideration of this type of information enables a richer and more accurate

-10-

assessment of the person and the likelihood of his achieving the target outcome. For example, a single mother may find it difficult to fulfil a job or course which requires a great deal of travel. Another example is that a student may have difficulties attending a course because they have to act as a full-time carer to a parent. Again, the information relating to 5 these attributes may be derived from a variety of sources.

Figure 9 provides an example of different types of attributes that may be incorporated into the system.

10 The motivation for profiling the individual is that the profile can then be assessed in relation to a desired goal, task or role (referred to as the 'target outcome'). The target outcome may be any type of pre-determined goal which the system user wishes to accomplish. Examples of target outcomes include aspirations such as 'complete a training course', or 'find employment', or 'select the most suitable person for a particular job/role/task'. In order to 15 assess the individual's suitability for the target outcome, or his potential ability to complete the target outcome, the person's profile is assessed in relation to a profile which is built to model the target outcome. Hereinafter this is referred to as the 'target profile'.

As with the person profile, the target profile is a set of attributes (i.e. data items) associated 20 with the target outcome. In other words, the target profile provides a list of criteria which are considered relevant to the target outcome. An example of the attributes or characteristics which might be included in a profile for a target outcome is shown in Figures 2 and 3. For example, the profile of a target outcome of 'select the most suitable person for a job vacancy' might include criteria such as 'possession of academic qualification X', or 'literacy 25 skills', or 'possession of a driving licence' and so on. Figure 2 shows an example of such a target profile. Figure 3 shows another illustrative target profile, in this case the target outcome is 'complete educational course X' (for example, a university course). A comparison of the example profile attributes given in these figures shows that the target profile is created with the particular target outcome in mind. The list of relevant profile 30 attributes is designed and selected by experts, possibly in consultation with interested parties (such as an education provider, an employer etc) who is in some way associated or familiar

-11-

with the target outcome. Thus, the attributes in the person profile may match or mirror those in the target profile.

The attributes contained in the target profile are associated with pre-determined 'scores' or 5 values. An attribute may be associated with one or more scores. The scores serve as threshold parameters and are used as assessment metrics when determining the person's likelihood of attaining the target outcome. The score for each attribute is pre-determined by the at least one expert. The expert(s) determine which attributes are relevant for the given target outcome, and also determine the score(s) associated with each attribute. This may be 10 done by consultation and/or collaboration with other experts and/or interested parties.

The score associated with an attribute may be a compulsory score or may be a preferred score. For example, possession of a driving licence may be a compulsory attribute for the target profile of the 'find the most suitable person for a car-based delivery vacancy' 15 outcome. During the assessment stage, the scores stored in the person's profile are compared against the compulsory/preferred scores in the target profile. If a person's profile has a value of 'no' for the 'possession of a driving licence' data item, then that person can be assessed as not suitable for this target outcome. (Conversely, if viewed from the person's perspective, the target outcome is not achievable for this person without appropriate up-20 skilling). Another attribute in the target profile may have a preferred score associated with it. For example, a particular target outcome may have an attribute relating to literacy level and a preferred person score of' 6'; however, a person with a literacy level score of' 5' in his profile may still be assessed as potentially suitable, especially if other scores in his profile exceed the target thresholds. This stage of the assessment and recommendation 25 functions of the system will be described in more detail below.

The functionality of the system can be viewed, at high level, as comprising the following processes or stages:

1. construction of the person profile and the target profile; and

30 2. assessment of the individual in relation to the target outcome; and possibly

-12-

3, generation of a recommendation regarding the upgrading (or' up-skilling') of the individual's attributes in order to achieve the target outcome or improve chances of success.

This overview of the system functionality is shown in Figure 4 and discussed as follows. 1. Construction of the person profile and the target profile

As described above, a person profile is designed by one or more experts who determine what the relevant attributes are.

The person profile is then implemented within a computer system. The profile is stored as a data structure which enables the scores to be stored (and subsequently accessed) in association with the respective person attributes to which they relate.

The person profile must then be populated with data. In other words, the scores (data values) for each attribute must be obtained and inputted into the system so that they are stored for future access.

The scores can be obtained in a variety of ways and from a variety of sources, using a variety of techniques, For example, the following methods may be used to obtain the necessary data pertaining to the individual:

• ' straight' evaluation - for example, a spelling test

• interview - question and answer process

• 'indirect' evaluation - for example, data regarding the individual's cognitive skills can be measured, deduced or inferred by their performance of a computer game

• External test results - data could be received and inputted into the system from a source external to the system. This data could be inputted manually, or could be received and/or processed electronically. For example, the individual could be asked to perform a physical test to measure attributes such as dexterity, balance, sight etc. The results obtained from the test could then be input into the system for storage in the person profile.

-13-

The methods and techniques used for obtaining the profile data are specified by the expert(s).

5 The type of data that is obtained will depend upon the attribute with which it is associated. For example:

• an interview question might be answered with a 'yes' or 'no' response, or might be some factual answer - for example, the answer to 'what is your postcode?';

• the person may select an option from a multiple choice question;

10 • a numeric result may be required - for example, if the person's mathematical skills are being tested, the data may be the answer to 'what is 8 + 7?';

• a person's prior knowledge such as spelling or open questions about facts;

• a person's cognitive or physical attributes may be measured by recording the person's responses to a test or the time taken to respond to a given test item; in some

15 cases.

As shown in figure 1, the data can be obtained from a variety of sources. The above examples illustrate some of the types of data which might be obtained from the person himself. However, data may also be obtained from other sources such as carers (e.g. family 20 and support workers), professionals (e.g. physician, psychologist, lecturer, teacher), employer (e.g. human resources department, line manager, colleagues) and so on.

However, the reliability of the data may vary depending upon the trustworthiness of the sources. For example, the person's view of their own skills might not be as reliable as 25 another person's opinion; or the person's opinion of their literacy skills may not be shared by his teacher. In many cases, for example, the objective opinion of a trained professional will be more reliable than the personal opinion of the person himself. In other cases, empirical data measured and obtained via actual tests may be considered to be more trustworthy than an opinion.

30

-14-

Thus, the attributes in a person profile are weighted to reflect the relative validity and reliability of the sources. This enables the system to provide a more accurate and informed assessment of the person's characteristics.

5 Initially, the weighting for each data source (e.g. the individual and the carer) will be assigned by the developers of the invention, using a combination of professional experience, consultation, and a review of available research in the appropriate field. From this a mean value may be derived. This mean may be adjusted in several ways. For example, the weighting of each data source may be adjusted as a function of the distance from the derived 10 mean. Tests of internal consistency will also be used to refine the values, such as comparing the individuals' self-perception of spelling skills to their actual skills as measured in a test. Feedback mechanisms will also be used to adjust weightings, such as the actual response of an individual in a given context compared to the expected outcome as noted by other evaluators. There may also be time related elements that have greater weight for recency.

15

In some scenarios, a conflict may arise where multiple sources give diverging opinions or data for a particular attribute. Artificial Intelligence (AI) techniques are employed by the system to resolve such conflicts. For example, multiple models may be implemented using different techniques, such as neural network techniques, Hidden Markov Models, Bayesian 20 rules, fuzzy logic and so on. Results produced by the different models can then be compared or evaluated. If the results converge, this can be interpreted as evidence of the reliability of the results.

Figures 6 and 7 show the use of a neural network in the context of a job application. With 25 respect to figure 6, each job may be profiled by an evaluator with a weighting depending on their knowledge and skill [w(e)] and by employees with respective weightings [w(i)]. (N.B. Error values are not shown.) Back propagation notes the distance between the weighted average and the individual result, with adjustments made relative to distance from everybody else's opinions.

30

Figure 7 illustrates how the value of the weighting may adjust according to the distance from the estimate in respect of the group average. This creates a dynamic, ongoing evaluation of

-15-

all values. This approach is beneficial for use in respect of profiling the target outcome and the individual.

Figure 8 shows a flow of sequences in a typical embodiment using a neural network in 5 profiling a job.

Examples of skills and abilities and other factors that form part of, and inform, the profile of an individual (i) may be as follows:

10 PRO(i,l) = Spelling ability

PRO(i,2) = Reading ability PRO(i,3) = Maths ability PRO(i,4) = Mental health PRO(i,5) = Location 15 PRO(i,6) = Dexterity

PRO(i,n) = N ability

The corresponding target profile is also constructed in the same way as described above. 20 The at least one expert determines which attributes will make up the target profile in consultation and/or collaboration with other parties. Thus, the data regarding the criteria/skills required for the target outcome may be provided by multiple sources. As with the person profile, the reliability of the data sources may not be uniform across all the sources. Thus, weightings are applied to the data obtained from the various sources to 25 address this issue.

Thus, to summarise, a job profile may be considered as a profile of outcome-related requirements derived from a series of sources, including expert job evaluators, managers and individuals already employed in task. The profile attributes derived by each of these will be 30 weighted according to the confidence of each of the expert evaluators. This will be iteratively derived using neural network techniques, whereby the target profile provided by any individual is compared with that derived by the rest of the group, and adjusted using

-16-

weightings that account for this difference. Furthermore, this value is updated by those in employment in that role, including any persons who may have been previously profiled in accordance with the invention in order to identify needs. In turn the ability of an individual to evaluate the skills required for the job they are in will be determined by their supervisor.

5

An example of a corresponding target profile is provided as follows, and including skills and abilities and other factors that form part of, and inform, the profile of a job or other target outcome 0

PRO(j,l) = Spelling ability 10 PRO(j ,2) - Reading ability

PRO(j,3) = Maths ability PRO(j,4) = Mental health PRO(j,5) = Location PRO(j,6) = Dexterity

PRO(j,n) = N ability

N.B. These values are example and do not correspond to actual components, since these are composite values shown for demonstration purposes only.

20 The system allows for any change that may occur in the individual, target outcome or up-skilling

2. Assessment of the individual in relation to the target outcome

After the attributes for the target and person profiles have been determined as explained 25 above, and the person data has been obtained and stored in the person profile, the assessment of the person's ability to achieve the target outcome can be undertaken.

The assessment is performed by comparing the attribute data (scores) stored in the person profile with the scores determined as required or preferred in the target outcome.

30

The ability of an individual (i) to do a job (j) may be expressed as follows For k = 1 to n

-17-

Skill_Difference(g,k) = PRO(i,k) - PRO(j,k)

If Skill_Difference(g,k) > 0

Then - No training required for assessed skills (k)

Else - Identify training for assessed skills (k)

5 Next

The assessment may be communicated to the person and/or other parties by a variety of 10 methods or techniques, and in a variety of formats. For example, the assessment can be provided as a report. The report may be provided in a form that is most suited to the needs of the individual, which may include written form, or audio form, for example.

3. Recommendation Regarding Up-Skilling

15 In certain embodiments, additional recommendation functionality of the system may be provided. The recommendation functionality of the system can be used to

• identify deficiencies or gaps in the individual's profile compared with the criteria in one or more target profiles;

• recommend a course of action which, if undertaken and completed by the individual, 20 could redress the identified deficiencies (and thus enhance the person's likelihood of achieving the target outcome)

The identification process may be accomplished by comparing the person profile against the target profile during the assessment stage. If one or more of the person's attribute scores fall 25 below the preferred or required threshold for that attribute as stored in the target profile, a strategy is recommended on how to bring the person's score for that attribute up to at least the pre-determined level stored in the target profile.

For example, if a target profile requires a score of '7' for mathematical skills but the person 30 profile contains a score of '4' for this attribute, the individual is identified as having a deficit of '3' for mathematical skills.

-18-

The recommendation may be, for example, a suggestion that the person

• takes an educational or other training course to obtain the desired skill and/or qualification;

• takes psychotherapy or other mental health treatment;

5 • undertakes a study skills training course to ensure their learning is maximised on their educational or training course;

• obtain a professional accreditation of skills acquired;

• considers alternative courses and/or career paths.

10 The recommendation is performed by drawing upon a database component containing information regarding up-skilling resources. For example, the database might contain details regarding educational courses, training courses and so on. Examples of the types of information contained in the database are shown in Figure 5.

15 The information stored in the database may be generic in the sense that it is not dependent upon or associated with any particular target outcome or person. However, in alternative embodiments the database may be tailored or custom-built.

Thus, the up-skilling resources that are known are profiled. The following provides an 20 example of skills and abilities and other factors that form part of, and inform, the profile of a up-skilling or training area (g)

Course entry skills (Course)

PRO(g,l,tO) = Spelling ability (e.g. literacy course)

25

PRO(g,2,tO) = Reading ability (e.g. literacy course)

30 PRO(g,3,tO) = Maths ability (e.g. maths course)

Course exit skills

PRO(g,i,ti) = Greater spelling ability

PRO(g,2,tl) = Greater reading ability

PRO(g,3,tl) = Greater maths ability

-19-

PRO(g,4,tO) = Mental health (e.g. mental health program)

(PRO(g,5,tO) = Location)

5 PRO(g,6,tO) = Dexterity

PRO(g,n,tO) = N ability

10

(not applicable)

(e.g. motor skills course)

PRO(g,4,tl) = Mental health PRO(g,5,tl) = Location PRO(g,6,tl) = Dexterity

PRO(g,n,tl) = N improved ability

The information stored in the person profile may be used to adapt the recommendation 15 provided by the system. For example, if the recommendation is that the individual upgrades his literacy skills, a literacy course held on the 2th floor of a building with no appropriate disabled access facilities is not suitable for an individual with restricted mobility. Also a vocational course that provides up-skilling in the construction industry, but requires Level 7 literacy skills but eh individual only has Level 5 clearly suggesting the course is 20 inappropriate without first obtaining literacy up-skilling. In such cases, the next best recommendation is computed and presented.

To identify best options for up-skilling (This assumes training for a job for which there is a vacancy. Integration of the Vacancies Database is another part of the system, though not a 25 pre-requisite for operation.)

Example scenario - To identify the training needs to move from the current skill base to the necessary skill base

1) Identify the Profile of the individual - PRO(i)

30 2) Identify the Profile of the job - PROQ

3) Identify the Profile of the training course - PRO(c)

NB This includes the entry requires and up-skilling offered.

-20-

4) Compare Profile of job with individual (Skill_Difference(g,k) = PRO(i,k) - PRO(j,k))

5) Identify a course that would provide up-skilling in key domains

For k = 1 to n

PRO(d,k) = PRO(c,k) - PRO(i,k) : PRO(d,k) =

5 Domain_Value_Added (k)

If PRO(d,k) < 0, Then PRO(d,k) = 0 : Since you cannot unlearn a skill

10 For k = 1 to n

PRO(w,k) = PRO(d,k) * w(j)

8) Ensure skills of the individual (PRO(i)) exceed the competencies required to carry out the 15 course.

For k = 1 to n

IF PRO(i,k) > PRO(g,k,tO): PRO(g) is course skills required THEN Course can be taken ELSE Insufficient entry requirements

20 Next

NB The above ignores for the purposes of this example factors such as resources needed, distance to course, time taken etc, all of which may be accounted for in the full model.

25 Full profile of an individual (i)

For k = 1 to n : n is all tests

PRO(i) = PRO(i) + PRO(i,k) : Score for each assessment of domains of an individual (i)

Full profile of a target outcome or job 0 For k = 1 to n

: n is all tests

-21-

PROQ = PRO(j) + PRO(j,k) : Score for each assessment of domains of a job G)

5 For k = 1 to n : n is all tests

PRO(c) = PRO(c) + PRO(c,k) : Score for each assessment of domains of a course (c)

For k = 1 to n : n is all tests

PRO(g) = PRO(g) + PRO(g,k) : Score for each assessment of domains of a course (g)

Example Implementations Of The Invention

The invention may be implemented in a computer system taking a variety of forms, as shown in Figure 10. Figure 11 shows an overview of the structure of an implementation in 20 accordance with the present invention.

For example, the system may be provided as an on-line facility wherein data is communicated across an electronic network such as the internet. In such embodiments, the data obtained from the individual and other parties is sent to a central server for storage over 25 the internet. The results generated by the invention (assessment and/or recommendation) may be communicated via the internet, for example via email or by presentation via a web site. Thus, centralised data management is made possible. Figure 10 illustrates some of the components which may make up an embodiment of the invention, and the various forms that the system may take,

30

Data management and structure allow data mining at different levels. The invention enables the implementation of a hierarchical structure to allow access to information at different

-22-

levels. For example, a teacher is able to view data for all of his/her pupils (but not other classes); a head teacher may see all classes but not other schools. Similarly there could be data access at local, district and national levels.

5 Data mining methods also enable multi-variant analysis, such as identification of spelling skills in Welsh first language speakers aged 21-30 with no formal qualifications, and a given disability.

The invention is also designed to be usable in any language. If the first or preferred language 10 of the individual is different to that of presentation of the material (Example 1 - there is a spelling test in English but the first language of the individual is, say, Gujerati or Welsh: Example 2 - the person answers they did not have certain support when young, because where they grew up they did not have that opportunity due to the cultural environment), the system will make recommendations with respect to all areas including language related 15 recommendations. The validity of this approach is maximised by using the system's databases to identify similar contexts and their respective responses to recommended actions. Thus, in the above example, the Welsh speaker is given information that may be significant due to the differences in their spoken and written language compared to English, which may be different again for Gujerati. Where the database does not provide sufficient 20 numbers to confirm and validate the recommendations, experts will be used to prime the system.

Other embodiments may provide off-line versions of the invention. In such systems, the data is not uploaded to a central, remotely located server but stored locally. The data is not 25 communicated via a wide area network.

In yet other embodiments, the invention may be provided as an 'intermittent' version of the system. This is used in situations where constant or reliable communications links are not available, for example in developing countries. In such implementations, the system 30 monitors network connectivity levels and transmits the data to the server when it determines that connectivity is sufficient to allow the transmission.

-23-

In further embodiments, an intranet version is provided. Thus, the system is implemented upon a closed network or limited access network. Such systems may be appropriate for certain applications such as use in prisons.

5 The system also provides the feature whereby feedback is continually incorporated such that, over time, the system is able to adapt, and performance is enhanced and refined. Therefore, the system provides a proactive and dynamic solution rather than a static, fixed, 'out of the box' solution as is the case with prior art systems. This means up-skilling of the individual, changes to jog specifications or training specifications can be updated and 10 comparative profiles re-evaluated.

Example Applications Of The Invention

The invention can be used with beneficial effects in a variety of areas and fields. The following are provided for purposes of representation and illustration only, and the invention 15 is not intended to be limited in this regard.

In some cases, the system may be arranged and configured for use in competitive situations. For example, a variety of person profiles may be developed and stored, along with a target profile for a particular job. In deciding which individual is best suited for the job, the 20 attribute scores can be compared across the person profiles to make an informed choice. Artificial Intelligence techniques may be used to weight the skills required, thus aiding the decision making process.

Alternatively, the system may be used

25 • A dyslexic student studying at university; his target outcome is to be able to complete the course; a profile is constructed of the student and the course, and the 'scores' from each are compared to identify any lack in skills or attributes which may be detrimental to the student's chances of success; a strategy may be recommended for upgrading any less-than-optimal attributes;

30 • Data collection for research - Collect large data samples, for example to help develop models of reading and inform teaching practice in minority languages.

-24-

• By an unemployed individual looking for a job (a plurality of target profiles may be drawn up for a plurality of available jobs; the individual's scores may be compared across the target profiles to find the optimal match i.e. the job which the individual is best suited for)

• Supporting or implementing government policies - For example, the impact of electricity being available at home for 14 year old students who have a minority language as their first language;

• In military training - Early identification of hidden difficulties - For example, motor difficulties (dyspaxia) would influence the instruction of individual for training of using bio-hazard protection suits;

• For employment placement - Identification of hidden disabilities to ensure inappropriate jobs were not suggested, or at least the caveats were provided;

• In university disability support - For identification of necessary study skills training required to support learning;

• In careers services - To help identify most probably routes to gainful employment;

• In personnel recruitment - To help placement of individuals, particular those with low skills in certain areas, to be appropriate placed.

Advantages of the present system include the following:

• The method and corresponding system enables a diverse set of characteristics to be factored into the assessment of the target outcome and individual; the characteristics can include positive and negative attributes, along with attributes relating to contextual information. By considering a richer variety of characteristics, the invention is able to provide a more accurate assessment of the individual's likelihood of achieving the desired outcome;

• provides a facility whereby any deficiency in the individual's profile can be identified, and a proposed course of action can be recommended to the individual which, if undertaken, would address the weakness;

-25-

• by using artificial intelligence, multiple sources of information can be used to develop appropriate analyses and recommendations. These include tracking individuals over time, ongoing review of recommended outcomes, comparison of individuals to others with similar backgrounds and target outcomes across large data

5 set, all of which help improve validity of any analysis and recommendation through the continuous use of feedback mechanisms.;

• the system is able to adapt and evolve over time as feedback and further information is added to the system; as this information is incorporated it enables the system performance to be refined and enhanced, and further comparisons made.

10

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be capable of designing many alternative embodiments without departing from the scope of the invention as defined by the appended claims. In the claims, any reference signs placed in parentheses shall not be construed as 15 limiting the claims. The word "comprising" and "comprises", and the like, does not exclude the presence of elements or steps other than those listed in any claim or the specification as a whole. In the present specification, "comprises" means "includes or consists of' and "comprising" means "including or consisting of'. The singular reference of an element does not exclude the plural reference of such elements and vice-versa. The invention may be 20 implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

25

-26-

Claims

CLAIMS:

1. A computer-implemented method for assessing a person's ability to achieve a pre-5 determined outcome, the method comprising the steps of:

constructing an outcome profile comprising a plurality of attributes which are 10 considered relevant to achievement of the outcome; and assessing the person's ability to achieve the outcome based upon a comparison of the attributes in the person and outcome profiles.

2. A computer-implemented method according to claim 1 and further comprising the

15 step of receiving a score for each attribute in the person and/or outcome profile and storing the received scores in association with their respective attributes.

3. A computer-implemented method according to claim 2, and further comprising the step of comparing the attribute scores in the person profile with the attribute scores

20 in the target profile.

4. A computer-implemented method according to claim 2 or 3 wherein the score in the outcome profile represents a predetermined threshold.

25

5. A computer-implemented method according to claim 3 or 4 and further comprising the step of computing a strategy for upgrading the identified score to at least a threshold level associated with a corresponding score in the outcome profile.

6.

30

A computer-implemented method according to any preceding claim wherein the profile attributes are weighted.

-27-

7. A computer-implemented method according to any preceding claim wherein the target profile includes at least one attribute relating to a disadvantage which adversely affects the person's ability to achieve the outcome

5

8. A computer-implemented method according to any preceding claim wherein at least one disadvantage relates to a physical, behavioural, mental health or learning disability cognitive disorder and/or psychological disorder exhibited by the individual.

10

9. A computer-implemented method according to any preceding claim wherein the person profile and/or the target profile includes at least one attribute relating to social, domestic or contextual factors.

10. A computer-implemented method according to any preceding claim wherein for 15 each attribute in the person profile there is a corresponding attribute in the outcome profile such that the attributes relate to the same features.

11. A computer-implemented method according to any preceding claim and further comprising the step of storing the attributes and/or scores in a form of computer-

20 readable memory.

12. A computer-implemented method according to any preceding claim wherein at least one attribute in the person and/or target profile relates to a skill, qualification or educational or vocational feature.

25

13. A computer-implemented method according to any preceding claim further comprising the steps of:

employing a variety of techniques, such as neural networking, Hidden Markov Models, Bayesian rules, to construct multiple models of the person and/or target 30 profiles; and comparing the results provided by the plurality of techniques.

-28-

A computer-implemented method for assessing a person's ability to achieve a predetermined outcome, the method comprising the steps of:

constructing a person profile comprising a plurality of attributes relating to the person, each attribute being associated with an attribute score;

comparing the attribute scores in the respective profiles; and generating a strategy for upgrading any attribute score in the person profile is less than the corresponding attribute score in the outcome profile.

A computer-implemented system comprising one or more components arranged and configured to execute the method of any preceding claim.