AU2007221955A1 - A Method for Creating a Psychological Scale - Google Patents

Description

S&F Ref: 821180 AUSTRALIA PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT Name and Address Canon Kabushiki Kaisha, of 30-2, Shimomaruko 3-chome, of Applicant : Ohta-ku, Tokyo, 146, Japan Actual Inventor(s): Joseph Anthony Thurbon Barry James Drake Address for Service: Spruson & Ferguson St Martins Tower Level 35 31 Market Street Sydney NSW 2000 (CCN 3710000177) Invention Title: A Method for Creating a Psychological Scale The following statement is a full description of this invention, including the best method of performing it known to me/us: 5845c(984470_ I) - 1 A METHOD FOR CREATING A PSYCHOLOGICAL SCALE Field of the Invention The current invention relates to creating a psychological scale and, in particular, creating a psychological scale from an experiment in which subjects are asked to make comparative judgements about multiple stimuli. 5 Background A psychological scale is a mathematical function that provides a number representing the psychological magnitude of each stimulus in some set of stimuli. For example, a set of stimuli might consist of a number of high quality printouts and these printouts may be placed on a psychological scale representing perceived quality. In other 10 words, each printout is a stimulus which someone may react to by judging its quality. The value of the printout on the psychological scale represents the expected quality judgement. Many researchers want to produce psychological scales for empirical data. Psychologists and sociologists would like to infer psychological scales to gain scientific insight into the minds of individuals and societies. Marketers would like to infer 15 psychological scales to predict how products will be perceived by markets. Engineers would like to infer psychological scales to work out the optimal configurations for equipment they design. Figure 1 shows four example stimuli, 1 to 4, and their locations on an example scale, X. The scale value of stimulus I is X[1], and so on. 20 Methods currently exist in the art that can produce a psychological scale from experiments, wherein the methodology of an experiment apportions particular pairs of stimuli (or collections of more than two stimuli) to each subject. 821180_specifinal 983554_19835541 -2 For example, in a "counting-based" method, each subject may be shown a collection of stimuli and asked to decide whether a particular feature or attribute that represents quality (or other factor) is present in each stimulus. From such data, a researcher counts of the number of times that each stimulus was perceived to have the 5 particular feature or attribute. From the counts, a psychological scale can be produced by following a process such as one of those reported in Psychometric Scaling: A toolkit for Imaging Systems Development, by Peter Engledrum, Imcotek Press, USA., 2000 (ISBN 0 9678706-0-7) hereinafter "Engledrum".. The "counting-based" methods require each stimulus to be tested and retested 10 many times to give reliable results. This can make the generation of a psychological scale too expensive. Also, the method is not appropriate when the quality factor under investigation cannot be expresses as the simple detection of a particular feature or attribute. Alternatively, in a "pair-based method", a collection of human subjects may be presented with pairs of stimuli and, for each pair, asked to pick the stimulus exhibiting 15 higher quality (or other factor). From such data, a researcher constructs a matrix, F, as illustrated for example in Figure 2A. The matrix F is formed such that F[i][j] is the count of the number of times that for the pair of stimuli, i and j, stimulus i was judged to have the higher quality (or other factor). From the matrix F, a psychological scale can be produced by following a process 20 such as one of those reported in Engledrum. For example, from the matrix F[i][j], another matrix D[i][j] can be constructed such that D[i][j] = F[i][j] - F[j][i]. An example of such a matrix D is seen in Figure 2B. D[i][j] represents a (signed) distance between i and j based on the frequency that i was judged to have higher quality (or other factor) than j. This distance matrix can then be 821180_specifinal 983554_1983554_1 -3 used to calculate a scale X, by choosing X such that the distances between stimuli according to X are as close as possible as distances between stimuli according to D. For example, X can be constructed such that the sum of the stress function (X[i] - Xj] D[i]j]) ^ 2, summed over all pairs of stimuli i and j, is minimised. 5 Figure 2C shows an example scale generated from matrix F of Figure 2A, according to the example definition of the stress function and the matrices D and F above. For "pair-based" methods, data needs to be collected for every possible pair of stimuli under investigation. If the number of stimuli to scale is N then the number of possible pairs of stimuli is N(N-l)/2. For example, given 10 stimuli there are 45 possible 10 pairs. For a large number of stimuli the number of possible pairs for which data needs to be collected becomes vast. For example, given 100 stimuli, there are 4950 possible pairs. Generally it is not practical to collect data for large numbers of pairs and so other methods exist in the art that can infer psychological scales without requiring data on all possible pairs. One such method is described in pages 101 to 102 of Engledrum. The cited 15 method starts by presupposing a scale (possibly derived from a pilot study). The researcher divides the set of stimuli into subsets that are contiguous and overlapping using the presupposed scale. Experiments are then run for each of the subset of stimuli. A scale is produced for each subset of stimuli. Therefore each subset's scale represents a portion of the complete scale. The cited method makes a complete scale by stitching together the 20 subset scales using the overlapping portions of each. The cited methods are based on ad-hoc solutions to the problem of producing a scale over a large number of stimuli. Consequently, they produce inferior results. There is the problem of the need to presuppose a scale in order to create the contiguous subsets. There is also a problem with determining the size of the overlap. A small overlap reduces 821180_specifinal 983554_1983554_1 - 4 the structural stability of the scale. That is, if the overlap is small, then when comparing two stimuli on the scale, the further apart they are on the scale, then the more unreliable is the comparison. The problem of structural stability can be ameliorated by increasing the size of the overlap, but this increases the number of stimuli pairs for which data needs to be 5 collected. Summary It is an object of the present invention to substantially overcome, or at least ameliorate, one or more disadvantages of existing arrangements. In accordance with one aspect of the present disclosure there is provided a method 10 for producing a psychological scale, said method comprising the steps of: (a) determining a number, F[i][j], of those times that a stimulus S[i] was preferred over a stimulus Sj]; (b) determining a number, FU][i], of those times that a stimulus Sj] was preferred over a stimulus S[i]; 15 (c) estimating a probability of preferring a stimulus S[i] over stimulus Sj] using (F[i][] + C/2) / (F[i][j] + F[j][i] + C) for some positive parameter C; and (d) forming the probabilities into the psychological scale for each stimulus pair [i][j]. In accordance with another aspect of the present disclosure there is provided a 20 method for producing a psychological scale, said method comprising the steps of: (a) estimating a distance, D[i][], between a pair of stimuli S[i] and Sj]; (b) estimating a weight, W[i][], to indicate the level of confidence in the distance estimate, D[i]b]; and (c) forming the weights into a scale for each stimulus pair [i][]. 821180_specifinal 983554_1983554_1 - 5 Preferably step (c) comprises finding scale values, X[1],...,X[N], for corresponding stimuli, S[1],...,S[N], that minimises the objective function: sum(W[i][] * (X[i] - X[j] - D[i][j]), i = I to N, j = I to N) by solving a linear system of equations. 5 In one implementation the solving of the linear system of equations uses Singular Value Decomposition. In accordance with another aspect of the present disclosure there is provided a method for producing a psychological scale, said method comprising assigning each stimulus, S[1],...,S[N], to groups, G[1],..., G[L], where the assigning comprises: 10 (a) assigning K stimuli into each group such that (i) each stimulus has an equal probability of occurring in the same group as any other stimulus, and (ii) considering all groups, no stimulus occurs more than once more than any other stimulus. 15 A computer readable medium having a computer program recorded thereon, the program being executable by a computer to perform the described methods, is also disclosed, as is corresponding computer apparatus. Brief Description of the Drawings At least one embodiment of the present invention will now be described with 20 reference to the drawings in which: Figure 1 shows an example set of stimuli on a scale, as known in the art; Figures 2A to 2C shows a scale X generated from a frequency matrix F; Figure 3 shows a flow chart describing a method of allocating stimuli to groups according to the present disclosure useful in the method of Figure 5; 821180_specifinal 983554_1983554_1 -6 Figure 4 is a schematic block diagram representation of a general purpose computer system upon which the method of Figure 3 may be implemented; and Figure 5 is a flowchart of a method of creating a psychological scale according to the present disclosure. 5 Detailed Description including Best Mode Presently disclosed are both a methodology for collecting experimental data and a method for processing the data to produce a psychological scale. The processing required to run the experimental methodology and produce the scale may be performed by any suitable processor. For example, all processing may be performed manually or by a digital 10 computer or a mixture of both manual and digital computation. An example of digital computer system upon which the presently described methodologies may be implemented is seen in Figure 4. The methods to be described with reference to Figures 3 and 5 may be implemented using a computer system 400 of Figure 4 wherein the processes of Figure 3 and 5 may be implemented as software. The software may include one or more application 15 programs executable within the computer system 400. In particular, the steps of the method of at least processing the data are effected by instructions in the software that are carried out within the computer system 400. The instructions may be formed as one or more code modules, each for performing one or more particular tasks. The software may also be divided into two separate parts. A first part and the corresponding code modules 20 may perform the psychological scaling methods and a second part and the corresponding code modules manage a user interface between the first part and the user. The software may be stored in a computer readable medium, including the storage devices described below, for example. The software is loaded into the computer system 400 from the computer readable medium, and then executed by the computer system 400. A computer 821180_sped_final 983554_19835541 - 7 readable medium having such software or computer program recorded on it is a computer program product. The use of the computer program product in the computer system 400 preferably effects an advantageous apparatus for creating a psychological scale. As seen in Figure 4, the computer system 400 is formed by a computer module 401, 5 input devices such as a keyboard 402 and a mouse pointer device 403, and output devices including a printer 415, a display device 414 and loudspeakers 417. An external Modulator-Demodulator (Modem) transceiver device 416 may be used by the computer module 401 for communicating to and from a communications network 420 via a connection 421. The network 420 may be a wide-area network (WAN), such as the 10 Internet or a private WAN. Where the connection 421 is a telephone line, the modem 416 may be a traditional "dial-up" modem. Alternatively, where the connection 421 is a high capacity (eg: cable) connection, the modem 416 may be a broadband modem. A wireless modem may also be used for wireless connection to the network 420. The computer module 401 typically includes at least one processor unit 405, and a 15 memory unit 406 for example formed from semiconductor random access memory (RAM) and read only memory (ROM). The module 401 also includes an number of input/output (1/0) interfaces including an audio-video interface 407 that couples to the video display 414 and loudspeakers 417, an 1/0 interface 413 for the keyboard 402 and mouse 403 and optionally a joystick (not illustrated), and an interface 408 for the external 20 modem 416 and printer 415. In some implementations, the modem 416 may be incorporated within the computer module 401, for example within the interface 408. The computer module 401 also has a local network interface 411 which, via a connection 423, permits coupling of the computer system 400 to a local computer network 422, known as a Local Area Network (LAN). As also illustrated, the local network 422 may also couple to 821180_specifinal 983554_1983554_1 - 8 the wide network 420 via a connection 424, which would typically include a so-called "firewall" device or similar functionality. The interface 411 may be formed by an EthernetTM circuit card, a wireless Bluetooth or an IEEE 802.11 wireless arrangement. The interfaces 408 and 413 may afford both serial and parallel connectivity, the 5 former typically being implemented according to the Universal Serial Bus (USB) standards and having corresponding USB connectors (not illustrated). Storage devices 409 are provided and typically include a hard disk drive (HDD) 410. Other devices such as a floppy disk drive and a magnetic tape drive (not illustrated) may also be used. An optical disk drive 412 is typically provided to act as a non-volatile source of data. Portable 10 memory devices, such optical disks (eg: CD-ROM, DVD), USB-RAM, and floppy disks for example may then be used as appropriate sources of data to the system 400. The components 405, to 413 of the computer module 401 typically communicate via an interconnected bus 404 and in a manner which results in a conventional mode of operation of the computer system 400 known to those in the relevant art. Examples of 15 computers on which the described arrangements can be practised include IBM-PC's and compatibles, Sun Sparcstations, Apple MacTM or alike computer systems evolved therefrom. Typically, the application programs discussed above are resident on the hard disk drive 410 and read and controlled in execution by the processor 405. Intermediate storage 20 of such programs and any data fetched from the networks 420 and 422 may be accomplished using the semiconductor memory 406, possibly in concert with the hard disk drive 410. In some instances, the application programs may be supplied to the user encoded on one or more CD-ROM and read via the corresponding drive 412, or alternatively may be read by the user from the networks 420 or 422. Still further, the 821180_specifinal 983554_1983554_1 - 9 software can also be loaded into the computer system 400 from other computer readable media. Computer readable media refers to any storage medium that participates in providing instructions and/or data to the computer system 400 for execution and/or processing. Examples of such media include floppy disks, magnetic tape, CD-ROM, a 5 hard disk drive, a ROM or integrated circuit, a magneto-optical disk, or a computer readable card such as a PCMCIA card and the like, whether or not such devices are internal or external of the computer module 401. Examples of computer readable transmission media that may also participate in the provision of instructions and/or data include radio or infra-red transmission channels as well as a network connection to another 10 computer or networked device, and the Internet or Intranets including e-mail transmissions and information recorded on Websites and the like. The second part of the application programs and the corresponding code modules mentioned above may be executed to implement one or more graphical user interfaces (GUIs) to be rendered or otherwise represented upon the display 414. Through 15 manipulation of the keyboard 402 and the mouse 403, a user of the computer system 400 and the application may manipulate the interface to provide controlling commands and/or input to the applications associated with the GUI(s). The method of forming a psychological scale may alternatively or additionally be implemented in dedicated hardware such as one or more integrated circuits performing the 20 functions or sub-functions to be described. Such dedicated hardware may include graphic processors, digital signal processors, or one or more microprocessors and associated memories. 821180_speci_final 983554_1983554_1 - 10 In order to place the presently disclosed methodologies into perspective, let S be the set of N stimuli such that S[1], S[2], S[3] and so on to S[N] are the stimuli. Let P be the set of M subjects such that P[1], P[2], P[3] and so on to P[M] are the subjects. The researcher is required to make a decision on the number of stimuli for which 5 each subject is capable of providing a comparison. Let K be the number of stimuli evaluated per subject. The methodologies may now be described as a series of eight steps, summarised by the method 500 in Figure 5. The method 500 has manual components, such as the obtaining of results from stimuli, however most steps can be implemented in an automated or computational manner with appropriate user /subject interaction. The 10 implementation may be facilitated upon the computer system 400, with the user/subject providing input or responses via the input devices 402, 403, and also receiving feedback or cues from the application program via the output devices 414, 417. Step 1: Step 501 of the method 500 operates to define a number of subject groups such that every subject is in at least one group and every group has at least one subject in 15 it. If there are a relatively small number of subjects (e.g., M < 100) then the researcher may decide that the number of groups is M, in which case every group contains exactly one subject. If the number of subjects is relatively large (e.g., M > 100), then the researcher may decide that the number of groups is a fraction of M and therefore each group contains multiple subjects. It is preferable that each group contains approximately 20 the same number of subjects. Step 501 may assign G to be the set of L groups such that G[1], G[2], G[3] and so on to G[L] are the groups. Step 2: Step 502 then operates to assign K stimuli to each group. The K stimuli are assigned into each group such that each stimulus has an equal probability of occurring 821180_speci final 983554_1983554_1 - 11 in the same group as any other stimulus, and considering all groups, no stimulus occurs more than once more than any other stimulus. However, other apportioning systems may be used. Every subject that is in a particular group will evaluate all of the stimuli assigned to that group. The method of assigning stimuli to groups directly influences the accuracy 5 of the inferred scale. At least one method for assigning stimuli to subject groups will be described below. Step 3: Step 503 operates to define an evaluation task to be given to the subjects. The evaluation task is used to determine a subject's preferred stimuli for each possible pair of stimuli in each group of which the subject is assigned. The task may include the subject 10 picking the preferred stimuli for each possible pair of a group. Alternatively the task may involve the subject ranking all stimuli in a group into a total order. From such an ordering, the preferred stimuli can be deduced for each possible pair. In a preferred implementation, the evaluation task requires a subject to form a total order over all stimuli in a group. The stimuli for example may be presented to the subject as visual stimuli via the video 15 display 414, or as audio stimuli via the loudspeakers 417, or both. Step 4: Step 504 of the method 500 operates for each subject to determine which group the subject is a member of, and therefore determine which stimuli the subject is to evaluate. Give the evaluation task to the subject and record the subject's preferred stimuli for each possible pair of stimuli. 20 Step 5: In step 505, for every pair of stimuli, S[i] and Sj], is evaluated for the subject's responses. Particularly, a number F[i][j] is determined that is a count of the number of times that S[i] was preferred over Sj], and a number F[j][i] is determined that is a count of the number of the number of times that S[j] was preferred over S[i]. 821180_specifinal 983554_19835541 - 12 Step 6: Step 506 operates to produce a signed-distance matrix, D. Each element, D[i][] of D is a signed distance between stimulus S[i] and stimulus S[j]. Positive distances indicate that S[i] is preferred over Sj] and negative distances indicate that Sj] is preferred over S[i]. In a preferred implementation, D[i][j] is determined using: 5 D[i][] = sqrt(2) * inverf (2 * Q[i][] - 1) where Q[i][] is the estimated preference probability given by: Q[i][j] = (F[i][j] + C/2) / (F[i][] + F[j][i] + C) and where inverf is the inverse error function defined by: inverf(erf(x)) = x 10 and where erf(x) = 2 / sqrt(pi) * integrate(exp(-t * t), t = -x to infinity). The parameter C represents the strength of prior belief that there is no difference between S[i] and Sj]. Traditional Thurston scaling, as described in Engledrum, does not use the parameter C, but setting C = 0 in the present method will produce the same 15 distances as traditional Thurston scaling. However setting C = 0 results in the well known problem of producing infinities that cannot be dealt with by known scaling methods. In a preferred implementation, the parameter C is some positive value, for example 1. The best value of C will be determined by the particular nature of the stimuli being rated, and the associated prior knowledge of those stimuli. 20 Other methods can be used to produce the signed-distance matrix, D. For example: D[i][j] = -signum(2 * Q[i]U] - 1) * log(l.0 - abs(2 * Q[i][j] - 1)). For another example: D[i][] = log(Q[i][j] / (1 - Q[i][j])). 821180_specifinal 983554_19835541 - 13 Step 7: Step 507 produces a weight matrix, W. Each element W[i][] of W indicates the level of confidence in the corresponding distance estimate D[i][]. In a particular implementation, W is calculated using: W[i][j] = (F[i][j] + Fj][i]) / (F[i][j] + F[j][i] + C) 5 In another implementation, W[i][j] can be determined using: W[i][j] = F[i][j] + FU][i] In yet another implementation, W can be determined using W[i][j] = 1. Note that not all scaling methods require a weight matrix. Consequently, step 7 10 can be skipped when a scaling method is used that does not require a weight matrix. Step 8: Step 508 produces a psychological scale, X, such that each stimulus S[i] has scale position X[i]. The preferred method finds the X values such that an objective function sum(W[i][j] * (X[i] - X[j] - D[i][j])A2, i = I to N, j = I to N) 15 is minimised. The objective function is minimised when the linear system of equations A-X = Y is solved. X is a column vector of the scale positions. A is a matrix defined by A[i]U] = -(W[i][j] + W[j][i]), when i does not equal j and A[i][i] = sum(W[i][k] + W[k][i], k = 1 to N) - 2 * W[i][i] 20 and Y is a column vector defined by Y[i] = sum(W[i][k] * D[i][k] + W[k][i] * D[k][i], k = I to N). The linear system can be solved using a variety of methods known in the art of Numerical Methods. In a specific implementation, the linear system is solved using a method known as Singular Value Decomposition. 821180_specifinal 983554_1983554_1 - 14 Thus the method 500 of Steps I - 8 has produced a psychological scale, X, for the stimuli S. In Step 2 (502) above, it was noted that the strategy for apportioning stimuli to groups directly influences the accuracy of the generated scale. The preferred method, 5 herein referred to as the "dealing method", apportions stimuli to groups in a manner that ensures that as the number of groups grows: (a) the number of times each stimulus is evaluated by a user converges to a single number, and (b) the number of times that each stimulus is compared to any other 10 stimulus converges to a single number. The dealing method is now described with reference to Figure 3, as seen in a flowchart 300. The dealing method 300 takes three parameters, a set of groups G[l] to G[L], a set of stimuli, S[l] to S[N] and an integer K, indicating how many stimuli are to be allocated to each group. There is an implicit constraint which is that N is greater than or 15 equal to K. That is, there must be at least as many stimuli available as there are stimuli allocated to each group. There are natural extensions to the method 300 wherein this constraint can be relaxed, for example, by creating physically identical stimuli when the experiment is carried out. These extensions are considered inferior to that presently depicted. 20 The dealing method 300 starts at step 310, where a list SP is initialised to an empty list. SP will contain, at any given time, a permutation of the elements of S which are to be distributed amongst G[1] to G[L]. Intuitively, SP can be thought of as a deck of cards, which is repeatedly shuffled and dealt from into to each group G[i]. 821180_specifinal 983554_1983554_1 - 15 Control then passes to step 315, where each of the groups G[1] to G[L] are set to be empty. By the time the method 300 terminates, each group will contain a subset of S. Control then passes to step 320, where i is set to 1. The variable, i, will be used to keep track of the group which is currently having stimuli allocated. 5 Control then passes to step 322. Step 322 has the purpose of making the flow chart simpler to comprehend. Control then passes to step 325. Step 325 determines if SP is empty. If it is empty, control passes to step 330, where the contents of S are copied into SP. Subsequently, control passes to step 335 where the contents of SP are permuted. Any 10 known standard permutation algorithm will suffice for this step. In the preferred implementation, the well known method of Knuth (also known as Fisher-Yates) shuffling is used. Control then passes to step 340. Returning to step 325, if SP is not empty, then control passes to step 340. At step 340, the first element of SP is removed from SP and stored in s, a 15 temporary storage location. Control then passes to step 345. Step 345 determines whether the current group G[i] already contains s. If the current group already contains s then, at step 350, s is appended to the end of SP, and control returns to step 340. In the case where the current group does not already contain s, at step 355 s is added to the current group, and control passes to step 360. 20 Intuitively, step 355 corresponds to "dealing" one stimulus into a group. This is because K will not necessarily be a factor of N, a group may have its stimuli dealt from two different permutations of SP. This requires that at step 345, a check is carried out to determine whether s is already in the current group. 821180_specifinal 983554_1983554_1 - 16 At step 360, a check is made to determine if the current group, G[i] has all of its K stimuli allocated to it. If G[i] has not had all of its stimuli allocated, control returns to step 325. If G[i] has had all of its K stimuli allocated to it, control passes to step 365. At step 365, i is incremented so that the next group can be processed, and control 5 passes to step 370. At step 370, a check is made to determine if there are any more groups to process. If there are (that is, if i is less than or equal to L) then control passes to step 322, where after the next group is processed. Otherwise, the method 300 terminates, and all groups have been allocated K stimuli. After the method 300 terminates, all G groups will be allocated K stimuli from S 10 in such a way as to maximise the accuracy of a psychological scale build using subjective evaluations from pairs of stimuli from within each group. Industrial Applicability The arrangements described are applicable to the computer and data processing industries and particularly for the assessment of human responses to different printer driver 15 configurations, print media or ink composition arrangements. These processes may be combined with a quality improvement process to inform the optimal driver settings, or other printer configuration parameters, for a desired market. The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and 20 spirit of the invention, the embodiments being illustrative and not restrictive. (Australia Only) In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including", and not "consisting only of'. Variations of the word "comprising", such as "comprise" and "comprises" have correspondingly varied meanings. 821180_spec_final 983554_1983554_1

Claims (8)

1. A method for producing a psychological scale, said method comprising the steps 5 of: (a) determining a number, F[i]U], of those times that a stimulus S[i] was preferred over a stimulus S[j]; (b) determining a number, F[j][i], of those times that a stimulus SU] was preferred over a stimulus S[i]; 10 (c) estimating a probability of preferring a stimulus S[i] over stimulus Sj] using (F[i]j] + C/2) / (F[i]j] + Fj][i] + C) for some positive parameter C; and (d) forming the probabilities into the psychological scale for each stimulus pair [i][]. 15

2. A method for producing a psychological scale, said method comprising the steps of: (a) estimating a distance, D[i][j], between a pair of stimuli S[i] and Sb]; (b) estimating a weight, W[i][], to indicate the level of confidence in the distance 20 estimate, D[i][]; and (c) forming the weights into a scale for each stimulus pair [i]U]. 821180 speci_final 983554_1983554_1 - 18

3. A method according to claim 2, wherein step (c) comprises finding scale values, X[l],...,X[N], for corresponding stimuli, S[1],...,S[N], that minimises the objective function: sum(W[i]U] * (X[i] - X[j] - D[i][j]), i = I to N, j = I to N) 5 by solving a linear system of equations.

4. A method according to claim 3 wherein the solving of the linear system of equations uses Singular Value Decomposition. 10

5. A method for producing a psychological scale, said method comprising assigning each stimulus, S[1],...,S[N], to groups, G[l],..., G[L], where the assigning comprises: (b) assigning K stimuli into each group such that 15 (i) each stimulus has an equal probability of occurring in the same group as any other stimulus, and (ii) considering all groups, no stimulus occurs more than once more than any other stimulus. 20

6. A method of forming a psychological scale substantially as described herein with reference to Fig. 3 or Fig. 5 of the drawings. 821180_specifinal 983554_1983554_1 - 19

7. A computer readable medium having a computer program recorded thereon, the program being executable by a computer to form a psychological scale as claimed in any one of claims I to 6. 5

8. Computer apparatus configured for forming a psychological scale, said apparatus being arrangement to implement the method as claimed in any one of claims I to 6. DATED this 11th Day of October 2007 CANON KABUSHIKI KAISHA 10 Patent Attorneys for the Applicant SPRUSON&FERGUSON 821180_specifinal 983554_1983554_1