WO2008122409A1

WO2008122409A1 - Trust manager and method for enhanced protection against spam

Info

Publication number: WO2008122409A1
Application number: PCT/EP2008/002692
Authority: WO
Inventors: Jimmy Mcgibney; Dmitri Botvich
Original assignee: Waterford Institute Of Technology
Priority date: 2007-04-05
Filing date: 2008-04-04
Publication date: 2008-10-16

Abstract

A method and apparatus for enhanced filtering of emails in particular to prevent dissemination of spam is described. The method relies on a distributed trust system, based on collaboration between separate mail transfer agents, to establish a trust level for different mail server nodes. The trust level is based on the mail transfer agents' experience of mail coming from that node. The trust level is then shared between the different mail transfer agents, and a composite trust level is then used to adjust how email coming from that node is processed by the system.

Description

Trust Manager and Method for Enhanced Protection Against Spam

Field of the Invention

The present invention relates to a trust manager and method for enhanced mail filtering on a mail transfer agent.

Background to the Invention

As well as being annoying, spam introduces many serious security risks and is often used to conduct fraud, as a conduit for malicious software and to carry out denial of service attacks on mail servers.

The principal current anti-spam techniques focus on filtering incoming email, either based on parsing message content, Bayesian filtering, maintaining Domain Name System (DNS) blocklists, and/or the use of collaborative filtering databases. Spammers tend to be resourceful though, and quickly find ways to get around most countermeasures. There are various attempts to also make anti-spam more adaptive, with the availability of reporting services that allow spam filters and blocklists to keep up to date with new spam techniques. There is a feeling though that spammers are always a step ahead, with the anti-spam community following with countermeasures some time afterwards.

In addition to technical countermeasures, several governments have introduced legislation outlawing spam and providing for stiff penalties. Despite some prosecutions, such legislation has had little or no effect, for several reasons such as the inter-jurisdictional nature of email traffic and spam.

Internet email is transferred using the Simple Mail Transfer Protocol (SMTP) (J. Klensin (ed.), Simple mail transfer protocol, RFC 2821, Internet Engineering Task Force, 2001). SMTP was first introduced (J.B. Postel, Simple mail transfer protocol, RFC 821 , Internet Engineering Task Force, 1982) at a time when the total number of Internet hosts was just a few hundred and trust between these could be assumed, and was designed to be lightweight and open. Attempts to introduce authentication via extensions or with new protocols that can sit on top of SMTP (e.g. Pretty Good Privacy, PGP - P.R. Zimmermann, The official PGP user's guide, MIT Press, 1995) have proven useful in some situations but are very far from universal adoption.

The Internet mail infrastructure, based on SMTP, has in general been very successful. Much of this success has been due to its simplicity. SMTP is a simple protocol wherein mail is transferred from a client system to a server system using a limited set of text commands. The focus of SMTP is on effective mail transport across heterogeneous systems, with the assumption that security or other desired features are provided at a higher layer or by means of extensions. The main security effect is in a lack of requirement for authentication of the source of an email or the path that it has taken.

The main anti-spam techniques in practical use are based on message content and DNS blocklists, and/or use of collaborative filtering databases. SpamAssassin (A. Schwartz, SpamAssassin, O'Reilly, 2004), for example, processes each incoming mail and assigns a "score" to it based on a combination of values attributed to possible spam indicators. The higher the score, the more likely it is that the mail is spam. A threshold is then used to filter mail - a mail that scores below the threshold is accepted and one that scores above the threshold is flagged as spam. The alternative, or complementary, approach is blocklisting - mail coming from unreliable sources is simply blocked.

These techniques have significant limitations though. With content filtering using a threshold, some genuine mail messages may score above the threshold and be flagged as spam (false positives) and some spam may score below the threshold and be accepted (false negatives). Careful tuning of the threshold can optimise the balance between the incidence of false positives and false negatives. For example, many email users will tolerate a modest level of spam to reduce the risk of some important mails being blocked. In practice, spammers are quite resourceful and adapt to content filtering advances - by, for example, using images rather than text, mutating text to avoid keywords that cause high filter scores, or spoofing source address to make it more acceptable.

DNS blocklists are also problematic. These effectively identify spam sources in a binary fashion (i.e. a mail source is either on the list or it is not). This becomes problematic when "good" mail transfer agents are attacked and exploited. Even with good system administration, this can happen and may not be noticed immediately. Though innovative new anti-spam techniques have been proposed, such as the use of micro- payments (J. Goodman and R. Rounthwaite, "Stopping outgoing spam", Proc. ACM Conference on E-Commerce, 2004) and a requirement for the sender to solve a computational challenge for each message, spam remains a significant problem for users and system administrators.

There has been some other work on applying trust and reputation information to spam filtering. Golbeck and Hendler (J. Golbeck and J. Hendler, "Reputation network analysis for email filtering", Proc. 1st Conference on Email and Anti-Spam (CEAS), 2004) present a technique based on social networks for sharing reputation information among email users. This allows users to sort received messages based on a sender rating value. This rating value is set by the user or is inferred from neighbours' ratings using a weighted average technique.

Kong et al (J. S. Kong, B.A. Rezaei, N. Sarshar, V.P. Roychowdhury and P. Oscar Boykin, "Collaborative spam filtering using e-mail networks," IEEE Computer, vol. 39, no. 8, pp. 67-73, Aug. 2006) also focus on end user email addresses, but provide for the anonymous sharing of information with a wider community. When a user flags a mail as spam, this is made available to other users' spam filters, which is useful as the same spam messages are usually sent to a large number of users. Damiani et al (E. Damiani, S. De Capitani di Vimercati, S. Paraboschi and P. Samarati, "P2P-based collaborative spam detection and filtering," 4th IEEE Int'l Conf. Peer-to- Peer Computing, 2004) similarly present a system of sharing spam information, wherein a cryptographic digest of each spam mail encountered is shared.

Seigneur et al (J. -M. Seigneur, N. Dimmock, C. Bryce and C. D. Jensen, "Combating Spam with TEA (Trustworthy Email Addresses)", Proc. 2nd Annual Conference on Privacy, Security and Trust (PST), 2004) use "hard" cryptographic authentication to support whitelists of known good guys in conjunction with "soft" trust management for new contacts. Trust derives from stored evidence, which includes recommendations, observations, certificates and reputations.

Foukia et al (N. Foukia, L. Zhou and C. Neuman, "Multilateral decisions for collaborative defense against unsolicited bulk e-mail", K. Stolen et al (Eds.): iTrust 2006, LNCS 3986, pp. 77- 92, 2006) are motivated to encourage mail transfer agents to restrict output of spam. Their approach is agent-based - each participating mail transfer agent has an associated Federated Security Context Agent that contributes to, and draws on, an aggregated community view (the Federated Security Context). They also use quotas to control the volume of mail output by a server in an attempt to prevent temporary traffic bursts that are typical of spammer activity.

It is an object of the invention to provide an improved system and method for the filtering of spam messages in a network.

Summary of the Invention

According to the present invention there is provided a method for providing enhanced mail filtering on a first mail transfer agent having a mail filter according to claim 1.

Preferred embodiments of the invention provide improved spam filtering, based on collaboration between mail transfer agents to manage trust. Each mail transfer agent records trust measures relating to each other mail transfer agent of which it is aware. Trust by one mail transfer agent in another is influenced by direct experience as well as recommendations issued by collaborating mail transfer agents.

Preferred embodiments of the present invention provide an architecture and protocol for establishing and maintaining trust between mail transfer agents. The architecture is effectively a closed loop control system that can be used to adaptively improve spam filtering. Mail transfer agents dynamically record trust scores for other mail transfer agents; trust by one mail transfer agent in another is influenced by direct experience of the server (i.e. based on mail relayed by that mail transfer agent) as well as recommendations issued by collaborating mail transfer agents. As well as enabling trust interactions between mail transfer agents, mail filtering according to the invention can combine trust values with existing mail filtering techniques.

Preferably, the protocol specifies how these experiences and recommendations are communicated between each spam filter and its associated trust manager, and between trust managers of different mail transfer agents. The protocol enables updating of a node's trust information about another node based on direct experience and referrals by third party nodes. This allows nodes to act autonomously and in a decentralised way. A set of related nodes with which a particular node exchanges trust information is referred to as its "neighbourhood". Defining this neighbourhood provides scalability and also reflects the ad hoc formation of groups and partnerships between nodes.

Use of distributed trust to tune spam filter parameters at mail transfer agents, in particular, spam thresholds, improves spam filter performance, in terms of (i) reduction in rates of false positives and false negatives and (ii) more efficient use of spam filter resources.

Use of sender trust level to determine how much processing power to apply to spam filtering, allows mail from less trusted sources to be subjected to more thorough tests and so provides more efficient use of spam filter resources.

Detailed Description of the Invention

An embodiment of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

Fig. 1 illustrates a system according to an embodiment of the invention in conjunction with a network of mail transfer agents; Fig. 2 illustrates the distribution of trust information in the system of Fig. 1 ;

Fig. 3 is a graph showing the comparison of dynamic vs fixed threshold systems, and the impact on the rate of false positives; and

Fig. 4 is a graph showing the comparison of dynamic vs fixed threshold systems, and the impact on the rate of false negatives.

Description of the Preferred Embodiment

Fig. 1 illustrates the relationship between SMTP and an architecture according to a preferred embodiment of the present invention. Mail transport operates as normal - it is not proposed to make any changes or extensions to SMTP or other existing mail protocols. With the trust overlay architecture, a trust management layer operates separately from mail transport. Two message passing interfaces are defined between the mail transport layer and the trust management layer and another between the trust managers of individual nodes. The interfaces are as follows (Fig. 1): (1) Experience reports: Mail host → Trust manager

(2) Recommendation: Trust manager <→ Trust manager

(3) Policy updates: Trust manager → Mail host

It is assumed, without loss of generality, that the Internet mail infrastructure is made up of a (large) number of SMTP-capable hosts. Each of these hosts can send and receive mail using SMTP. For the remainder of the description, the hosts are referred to as nodes.

Trust is defined as being between two nodes - i.e. node A has a certain level of trust in node B. Each node records a set of trust-related parameters for each other node, hi the general case, there could be many such parameters corresponding to a variety of services for which trust measures are used. Trust in another mail transfer agent is preferably represented by a score in the range (0,1) - each node records this score for each other node of which it is aware. A trust score of 0 means that there is no trust in this node and a trust score of 1 means that this node is fully trusted. Some trust scores will be very stable and may result from plenty of experience and corroboration by several peers and thus there may be a high level of confidence in these scores. Other trust scores may be less reliable. Thus a confidence level for each trust score is also recorded, again in the range (0,1). A third parameter that is used is recency, indicating how recently this trust score was updated. Recency is important as mail transfer agents come and go; good mail transfer agents become bad (e.g. due to a hijack) and bad mail transfer agents become good (e.g. due to removal of malware). Recency can be recorded in any appropriate units of time.

This can be implemented as a table that might look something like that shown in Table 1 :

Node name Trust Confidence Recency score smtp2.foo-inc.com 0.99 0.99 100 mail.barfoo.org 0.95 0.95 200 labserver.uxy. edu 0.80 0.80 700 webmailprov.com 0.50 0.50 30 lazyconfig.net 0.25 0.25 10 phishysitel 342.com 0.01 0.01 8000

Table l. Representation of trust scores, from perspective of a specific node (smtp1.foo-inc.com)

Nodes can control the size of this table by choosing to "forget" a trust score and deleting its entry. For scalability reasons, trust scores could just be recorded for nodes that have been encountered fairly recently.

Trust scores are recorded per IP address (IPv4 or IPv6). Table 1 shows hostnames only for reasons of readability. A trust score thus is just meaningful when applied to mails from the IP address to which it is attached. As SMTP does not protect against modification of sending SMTP server hostname or IP address by intermediate mail relays, processing of trust values is always based on the last hop. Mail received from an SMTP relay affects the trust score of that SMTP relay rather than the originating host. This provides the relay with a strong incentive to filter outgoing spam. The IP address of the last hop is collected from the source IP of packet(s) containing the SMTP HELO/ELHO message (to which a reply will have been sent).

The architecture described here is highly distributed, with each node autonomously managing its own view of the trustworthiness of other nodes. In practice, however, some nodes might prefer to defer to a centralised trust reference. Such a centralised trust reference can be modelled as a virtual node with a very high trust value (perhaps 1.0).

A protocol is now described which is referred to as TOPAS (Trust Overlay Protocol for Anti Spam). The TOPAS protocol is for collecting spam statistics to calculate trust, sharing this trust information between nodes and feeding it back to mail hosts to allow them to more effectively filter spam. The protocol executes using asynchronous message passing in the case of interfaces (1) and (2) of Fig. 1. Interface (3) uses a simple synchronous request-reply technique. An underlying set of services is assumed, including message delivery, failure detection and timeout. It is assumed that each process receives queued messages of the form (tag, Argl, ..., Argn). This outline style of protocol specification is influenced by the Generic Aggregation Protocol (GAP - M. Dam and R. Stadler, "A generic protocol for network state aggregation", Proc. Radio Science and Communication conference (RVK), Linkόping, 2005).

(1) Experience report: Mail host — > Trust manager The mail transfer agent has an associated spam filter. Each item of mail that arrives is processed by the spam filter, with the result that it is either accepted or flagged as spam. This information needs to be available to the trust manager. The following two messages, from the mail host to the trust manager, provide this:

• (singleMail, i, s) may be sent from the local mail host to the trust manager to report on a single mail from node i. The value of s is 1 if the mail has been determined by the mail host to be spam and 0 otherwise, i identifies the sending node.

• (bulkMail, i, n, s) may be sent from the local mail host to the trust manager to report on a sequence of mails from node i. n is the number of mails received from node i (since the last report) and s the number of these determined by the mail host to be spam.

Note that in both messages above, the node identifier i could be an IP address, a hostname, a mail domain name or any other identifier supported by the mail host. It is up to the trust manager to process this.

Sending a bulkMail message conveys the same information as would several singleMail messages. It is included in the protocol for performance reasons. Where mail volumes are heavy, it is more efficient to send periodic bulkMail updates rather than burden the mail transfer agent with generating a singleMail message per mail received.

(2) Trust recommendation: Trust manager <→ Trust manager Nodes collaborate to share trust information with one another. This is done by the trust managers, using the primitives outlined here. Trust managers may issue recommendations spontaneously (for example on occurrence of some event like sudden appearance of spam) or in response to a request from another node. Note that a request may be issued by one trust manager to another, but a reply is not guaranteed. As mentioned earlier, message passing is asynchronous and the protocol requires no state information to be maintained by the corresponding entities. The following six messages may be sent from one trust manager to another:

• (getTrust, k) allows node i to ask another node j to report its trust in node k. This message should be interpreted as an indication from node i of a desire to receive a recommendation regarding node k. Node j may respond with a trust report. This might be expected to be used as follows. On receipt of some volume of mail from node k, node i could issue a series of getTrust messages to neighbouring nodes requesting recommendations regarding node k. Only those neighbours with some experience or knowledge of k might reply, with the remainder staying silent. Note though that nodes are not obliged to respond, even if they do have knowledge or experience of k. Nodes could have other reasons (e.g. lack of trust in the requester, i) to not reply. Nodes that have no information are expected to stay silent.

• (getTrustResponseRequested, k) is a variation on getTrust that expects the recipient to respond (with a null value) even if it has no trust information on node k. Again, there is no strict obligation to respond.

• (getTrustAll) allows one node i to ask another node j to report its trust in all known nodes. This message should be interpreted as an indication from node i of a desire to receive a recommendation regarding all nodes of which the recipient is aware. There is no obligation on the recipient to respond. Nodes that have no information on any nodes should issue a null reply.

• (setTrustReportingPreferences, t, c, r, f) allows node j to specify to node i how spontaneous trust reports are sent to it (see discussion on "push" model later). This message also indicates a desire by node j to receive trust advertisements from node i (i.e. to be included in its neighbourhood). t is a list of zero or more trust level thresholds. Trust updates are requested whenever trust exceeds or falls below any of these threshold values. c is a confidence level threshold. Trust updates are only desired if the confidence level of the sender is at least c. r is a recency threshold. Trust updates are only desired if the trust information has been updated by the sender within the previous r time units. f indicates the maximum frequency of update.

• (trustReport, k, T) allows node j to send a recommendation to another node i, in relation to node k. T is an object that encapsulates sender j's trust in node k. T is set to null in the case where the sender j has no trust information regarding node k. Note that a trustReport may be issued either spontaneously or in response to a getTrust or getTrustResponseRequested message.

• (bulkTrustReport, 1) allows node j to send a recommendation to another node i, in relation to a set of nodes. Parameter 1 is a set of pairs (k, T) where k is the node identifier and T is an object that encapsulates sender j's trust in node k. 1 is the empty set in cases where the local node has no trust scores to share. Note that a bulkTrustReport may be issued either spontaneously or in response to a getTrustAll request.

(3) Policy update: Trust manager -→ Mail host

The third part of this collaboration architecture is responsible for closing the loop. Direct experience is recorded by nodes and shared among them. The result of this experience and collaboration is then used to inform the mail host to allow it to operate more effectively. Specifically, the mail host, on receiving a mail from node i needs to be able to access the current trust information that its trust manager has on node i. Although the mail host may be able to use local storage to, for example, cache trust values, we do not place any such requirements on it. Thus it is needed to provide the ability for the mail host to request trust information from the trust manager and receive a timely reply.

• (getTrustLocal, i) allows the mail host to request the trust score for a particular node, i.

• (trustReportLocal, i, T) allows the trust manager to respond to a getTrustLocal request. T is an object that encapsulates the trust manager's trust in node i. T is set to null in the case where the trust manager has no trust information regarding node i.

The Internet mail architecture is highly dynamic - new nodes appear all the time, and existing nodes will quite frequently receive mail from previously unknown senders. In distributed trust management generally, choosing an initial value of trust to assign to such new arrivals is a non- trivial task. There are essentially two options, where the range of trust is (0,1):

(1) Initialise the trust level at zero. (2) Assume some "default trust" exists and initialise at some value above zero.

Treating previously unknown senders the same as spammers would restrict the utility of email as it tends to block new entrants; this points to the second option as perhaps the best.

The functions specified at interface (2) above allow for either a "pull" or a "push" model for sharing trust information. Messaging is asynchronous for experience reports and recommendations .

"pull" model

The trust manager receives a getTrust, getTrustResponseRequested, or getTrustAU request for trust information about another node, or all nodes, and subsequently replies with a trustReport or bulkTrustTeport message. The sender of the request will need to use its own timeout mechanism. In the case of a getTrust message, there is no obligation on the recipient to reply at all. With getTrustResponseRequested and getTrustAU, the recipient should issue a reply even if this contains no trust information.

"push " model

The trust manager may be configured to spontaneously issue trust updates, either to other nodes or to its local mail transfer agent. This is typically for reasons of performance and efficiency. It is wasteful for nodes to repeatedly poll each other unless there is useful new information, hi the "push" case, each node decides when and to whom to issue trust advertisements.

Each individual node controls the frequency of issuance of trust advertisements, and may even decide to issue none. Issuing a trust advertisement uses processing, memory and network resources of both the sender and recipient. The sender needs to find a balance - the more collaboration the more effective the system, but sending too many updates may put a strain on resources. It is also desirable if the recipient can control the number or frequency of advertisements from the sender. A sender trust advertisement strategy might have them sent periodically or on significant change in trust value or confidence level in this value. For example, the sender may issue trust advertisements relating to a node when its trust in that node exceeds a certain threshold and when its confidence level in this trust is high. If the trust level exceeds the threshold but its confidence level in this value is low, it may choose not to send anything. The TOPAS protocol allows this to be based on the recipient's specified trust reporting preferences.

Each individual node also controls the set of nodes that forms its "neighbourhood" - those nodes to which it sends trust advertisements. The set of neighbours could be defined, for example, as containing nodes that are nearby, most trusted, most collaborative, and/or have specifically requested trust reports (using setTrustReportingPreferences).

Fig. 2 illustrates the distribution of trust information. There are eight nodes (mail transfer agents) in the example shown. Each node has its own neighbourhood, two of which are shown. In some short time duration, node 'F' sends mail to 'A', 'D' and 'E'. Node 'A' establishes a trust score for node 'F' based on (i) mail it receives directly from 'F' and (ii) reputation information from its neighbours, 'B', 'C and 'D'. The reputation information provided by node 'B' relates the experience of node 'E' that 'E' has shared with it. The reputation information provided by node 'C derives its trust score relates the experience of node 'D' that 'D' has shared with it. Note that trust transitivity may allow a trust score to propagate quite some distance - in this example, the score that 'E' records for 'F' is propagated to 'B' and onward to 'A'.

With the above architecture, each node can implement a model for using trust information to enhance security through collaboration with other nodes. Note that nodes might implement a different model for updating and using trust, and it can also be expected that a node's behaviour in terms of handling trust may change if it is hijacked.

One of the problems of modelling decentralised systems like ad hoc networks is that it is unrealistic to take a top-down "bird's eye" view. Thus, in preferred embodiments of the invention, the set of nodes that are modelled are those nodes of which a specific node is aware.

It is therefore assumed that neighbour discovery and routing services are in place. It is also assumed that some kind of service registration and discovery is available to allow nodes to reach an understanding of the set of services available and their associated trust thresholds. It should be noted that the assumption of a common threshold across all nodes that provide the same service is something of a simplification. Even having all nodes share an understanding of the relative meaning of trust threshold values is non-trivial. Authentication of identity is also assumed. This could be done, for example, by having nodes exchange public keys on their first interaction (in fact the public keys could be used as unique node identifiers). Further messages between those nodes could then be signed by the sending node's corresponding private key so that the recipient could at least be confident that the sender is the same entity as that for which it has been building up a trust profile.

Both the way trust is updated based on service usage, and how referrals are issued and handled, have a profound impact on the usefulness of this kind of trust overlay system. The best models depend on network topology, node mobility, the range and types of service on offer, the attack threat model including attacker collusion scenarios, the potential proportion of "bad guys" in the system (i.e. compromised nodes) and security requirements. How well models work can be assessed in terms of system stability and improved resistance to attack.

Trust update models implemented by nodes in accordance with the present invention are described using the following terminology:

Topology. Let V₁ = {I,... ,N} be the set of nodes of which node i is aware. Some of these nodes will be adjacent to i, normally by reason of network topology. This can be modelled by an adjacency vector, A = (α,_,7) _{=1 N} , where a_{t J} = 1 if pair (ι,j) are neighbours and a_{l }} = 0 otherwise.

Services: Let S = {s_u...,S_M } be a set of services that may be provided. Each node j provides a set of services S^J cz S . Some nodes will just be service consumers, so S^J will then be the empty set.

Trust Each service S_r has an associated trust threshold t_r . thresholds:

Representing We denote the local trust that node i has in node j as T₁₁. Each other

Trust'. _no(j_e £ ₆ p. _w^j maintain it_s own local view of trust iny, which may, as we shall see, influence the local trust of node / in j. Note that T₁₁ is a trust vector, hi the simplest case, this can just a number, but such a number may be associated with other attributes that relate to it, like confidence in the trust score or recency.

Trust In the case where no trust exists between the nodes, we will have initialisation'. T , =χ where 0 < x ≤ 1 is the default trust value. Trust When node/ attempts to use service S_x provided by node /: decision: • Service use is permitted if f(T_hJ) > t_x , where the function/maps trust (using trust in service vector T,_j onto a scalar number in the range (0,1) protection) • Otherwise node/ is blocked from using service S_x

Trust update After a service usage event, by node/ on node i: following • positive outcome => T_{1 1} increased according to some algorithm service usage'. • negative outcome => T_{1 }} reduced according to some algorithm In general, following service usage, we update T₁₁ according to:

T_tJ <- f_e{T_hJ,E) (1) where E is a vector of attributes related to the service usage event and f_e is a function defining how trust is updated based on service usage experience.

Trust update Node i may receive a message from a third party node, k, indicating a following level of trust node/. This can be modelled as node / adopting some of recommennode A:' s trust level in node/, T_{k j} . hi general, following such a third dation by a party recommendation, we update T₁₁ according to: third party:

T,_tJ ^ f_r{T_tiJ ,T_l<k,T_kJ (2) where f_r is a function defining how trust is updated. This trust transitivity depends of course on T_{1 k} , as node i can be expected attach more weight to a recommendation from a highly trusted node than one from a less trusted node.

Models implemented by various nodes in accordance with the invention include combinations of:

• Moving average: Each new assessment of trust is fed into a simple moving average, based on a sliding window. Direct experience can be given more weight than third party referrals in the averaging if desired. More advanced moving averages are also possible, where old data is "remembered" using data reduction and layered windowing techniques.

• Exponential average: Each new assessment of trust is fed into a simple exponential average algorithm. Exponential averaging is a natural way to update trust as recent experience is given greater weight than old values, and no memory is required in the system, making it more attractive than using a moving average. Direct experience can be given more weight than referrals by using a different (higher) exponential averaging parameter. In our initial simulations, we update trust based on experience as follows:

where parameter α, 0 < a ≤ 1 , defines the rate of adoption of trust based on experience. Note that having a = 0 means that the trust value is unaffected by the experience. Having a = 1 means that local trust is always defined by the latest experience and no memory is retained. In general, the higher the value of α, the greater the influence of recent service usage on the trust value maintained for that node. Lower values of a encourage stability of the system. In our initial simulations, we update trust based on referrals as follows:

^Ti,J *^{~ T}i,j ^~ β^Ti,k V i,j ^{~ T}k,j ) (4) where β is a parameter indicating the level of influence that "recommender trust" has on local trust, 0 < β < 1 . Note that, the larger the value of T_{1 k} , (i.e. the more i trusts k), the

greater the influence of k's trust in j on the newly updated value of ^tJ . Note that, if

Ti ]₁ = 0 ₅ this causes * i,j to be unchanged.

• Exponential average, per service parameters: Parameters depend on service. Here, some services are more revealing than others of user's trustworthiness. Usage of a service that provides little opportunity for exploitation (say a service that allows read-only access to a system) shouldn't have much impact on trust in the user.

• Exponential average, with different parameters for trust increase and decrease: Here, one exponential averaging parameter is used for increasing trust and another for reducing it.

• No forgiveness on bad experience: This in a draconian policy: a node behaving badly has its trust set to zero forever.

• No forgiveness at all: (experience or referral). Even more extreme: a node that is reported by a third party, or observed locally, as behaving badly has its trust set to zero forever.

• No forgiveness, for certain services: Apply a draconian policy if a particular sensitive service is exploited. • Second chance (more generally, nth chance): Draconian policies are generally not a good idea. Intrusion detection and other security systems are prone to false alarms. Benign activity can occasionally be flagged as malicious. Good nodes can be temporarily hijacked and should be given the opportunity to recover. Thus a suitable variation on the "no forgiveness" algorithms is to keep a count, perhaps over a sliding time window, of misdemeanours. Trust is set to zero, possibly forever, on n misdemeanours.

• Hard to gain trust; easy to lose it: To discourage collusion between bad nodes, there is a case for making it hard to gain trust and easy to lose it. Thus attackers will require lots of effort to artificially increase their trust scores, either by repeated benign use of low- threshold services or by issuing repeated positive recommendations about one another. Comparatively little malicious activity will cause trust to fall significantly.

• Use of corroboration: To prevent an attack by up to k colluding bad nodes, we could require positive recommendations from at least k+1 different nodes.

• Use of trust threshold for accepting referrals: It is possible to model the ability to issue a referral to a node as a kind of service usage on that receiving node. Thus the receiving node can apply a trust threshold to decide whether to accept that referral in the same way as any service usage attempt is adjudicated.

As already indicated, collaboration by nodes within the system operates by issuing of trust referrals. Certain rules are required to make this system workable in possibly large scale distributed environments. Some approaches include:

• Random destination and frequency: At random intervals, a node i chooses a random other node that it knows about, j, to which to issue a referral. This approach ensures even node coverage over time, and scalability can be ensured by agreeing common upper bounds on the frequency of referrals. • On request: Node i only issues a referral to node j when asked for it. From node j's perspective, there is no guarantee that node i will respond. • On service usage experience: Node i only issues a referral relating to node k just after a service offered by node i has been used by node k. This means that the referral is based on fresh experience.

• Only on reduction in trust: Referrals are only issued on detection of malicious activity, hi this case, the trust system would act like part of a distributed intrusion detection system

(IDS).

• Each node specifies own trust receiving preferences: Here, each node specifies how often and under what conditions spontaneous trust reports are sent to it. A node may only wish to receive referrals if trust has changed significantly, for example.

Using a given model as outlined above, a mail transfer agent applies spam filtering to each incoming email. In each case, a decision is made whether to accept the mail preferably by comparing a score with a threshold. A negative result (in the spam test) means that the mail is accepted and a positive result means that the mail is rejected (or at least marked as spam).

Preferably, the threshold is allowed to vary and depends on the trustworthiness of the node that sent the message. So, the sender's trust score (as perceived by the receiver) is used to define the threshold — i.e. the more trusted a node is, the higher the threshold is set for marking a new mail message as spam. Conversely, if a node is untrusted then the threshold is set to a lower value.

There are several ways to cause this threshold to vary. For example, the threshold for mail received from a node could be simply a linear function of the trust score for that node. For example, the default threshold for a mail filter such as Spam Assassin to reject a mail message is 5. The mean of a trust score range is 0.5 and so the threshold could be set to be simply ten times the trust score for the node from which mail is received.

hi practice, the dynamics of trust applied to spam filtering allows an organisational mail transfer agent to process email in a way that depends on its trust in the sending node, hi many cases, this trust level will be somewhere in the middle, between "trusted" and "untrusted".

hi a fixed threshold situation, where the spam filter threshold is 5.0, based on a Spam Assassin implementation, if a genuine messages scores 5.1, it is diverted to the spam box, and if a spam message scores 4.9, it is allowed into the user's inbox. If instead there is a dynamically tuned threshold, however, a genuine message from a trusted source scoring 5.1 might be accepted as the source should have a higher threshold. Likewise, spam received from an untrusted node scoring 4.9 might be filtered out as the threshold should be lower.

An illustrative example showing a possible implementation of the TOPAS protocol is now shown, having a simulated network of mail transfer agents generating traffic, some of which is spam. It is shown how improvements in spam filtering (in terms of reduced false positive and false negative rates) can be achieved through closed loop control based on sharing trust scores.

hi the simulations, each node has a neighbourhood defined. This is a set of nodes that are somehow "close" to the node in question, with the expectation of above average frequency of communication with them.

Neighbourhoods are defined randomly for each node. This is done as follows: Initially, the neighbourhood of each node is the empty set. Choose two nodes at random. Add one to the neighbourhood of the other. Repeat until the total set of neighbour relations is equivalent to a connected graph. Then trust will (eventually) propagate throughout the network.

Email traffic volumes also vary randomly, with spammers tending to produce email in greater quantities than regular email users. For each mail sent, the recipient can be anywhere on the network, but is more likely to be a neighbour than any random node.

Each email contains a value S' that models aggregated indicators of spam, in the style of SpamAssassin. This value is used by the receiving node to test for spam.

S' has a Gaussian (normal) probability density function with mean μ and standard deviation σ. Mail that is actually spam tends to have a high value of μ. Normal mail tends to have a lower value of μ. The standard deviation determines the tendency for the filtering system that analyses such mail to be prone to false positives and false negatives.

Whether or not an email is actually spam is indicated by a binary value that is communicated separately to the receiver. This is not used in spam detection, but is used afterwards in the evaluation of how well the spam filter worked. Two different approaches to mail filtering are examined:

• Use of a pre-defined threshold. The value of the threshold is selected to be halfway between the means of the probability density functions of S' for spam and non-spam respectively. Preferably, this mean is 5.0 (a common SpamAssassin threshold).

• Use of an automatic threshold that is directly related to trust in the sending node.

The first approach takes no account of trust information. The second uses trust information to tune the spam filter.

It is now illustrated how improvements in spam filtering (in terms of reduced false positive and false negative rates) can be achieved through closed loop control based on sharing trust scores. A network of fifty nodes is simulated, of which a single one is a spammer. The spammer is responsible for 50% of all email generated in the system. Trust convergence for "good" nodes is based on exponential averaging, with a parameter of 0.03 (chosen experimentally for stable but moderately fast convergence). The randomly generated neighbourhood of each node comprises on average of one-seventh of all nodes.

Relatively flat (but distinct) probability density functions are chosen for spam indicators for both spam and non-spam email. Both have the Gaussian (normal) distributions shown in Table 2. Note the overlap implied by the relatively large standard deviation values.

As already mentioned, most spam filters combine a variety of measures into a suspicion score and compare this score with a pre-defined threshold. Preferably, a fixed threshold of 5.0 is chosen (SpamAssassin default) and used as a benchmark. As can be seen in Figs. 3 and 4, a significant reduction in both false positives and false negatives can be achieved with auto-tuning of the threshold (based on trust values). Auto-tuning is of course most effective in a steady-state situation when trust values are quite stable. A range of other predefined threshold values were also tried, but with no better results than the value of 5.0 shown. Choosing a higher predefined threshold causes an increase in false negatives and choosing a lower predefined threshold causes an increase in false positives. Table 2. Parameters for Gaussian (normal) distributions used in experiments

Specific spam filtering techniques are not of direct concern and can effectively be plugged in as needed. The emphasis is on using trust information to tune such filters, and the initial implementation focuses on filters that are based on thresholds.

The invention is not limited to the embodiment described herein but can be amended or modified without departing from the scope of the present invention.

Claims

1. A method for providing enhanced mail filtering on a first mail transfer agent having a mail filter, the method comprising the steps of: (a) responsive to filtering of at least one email received from a first remote mail transfer agent using said mail filter on said first mail transfer agent, generating a first recommendation for said first remote mail transfer agent based on said filtering;

(b) forwarding said first recommendation from said first mail transfer agent to at least one collaborating mail transfer agent; (c) receiving from a collaborating mail transfer agent at least one recommendation in relation to a second remote mail transfer agent; and

(d) adjusting the mail filter on said first mail transfer agent in relation to emails received from said second remote mail transfer agent based on said at least one received recommendation.

2. The method as claimed in claim 1, further comprising the step of adjusting the mail filter on said first mail transfer agent for further emails received from said first remote mail transfer agent, based on said filtering of said at least one email.

3. The method as claimed in claim 1 or 2, wherein said recommendations are trust recommendations.

4. The method as claimed in claim 3, wherein said step of generating comprises: receiving a notification from said mail filter that said at least one email from said first remote mail transfer agent is spam; maintaining a trust score for said first remote mail transfer agent, based on a function of the percentage of emails received from said remote mail transfer agent that are spam and based on any recommendation received from a collaborating mail transfer agent in relation to said first remote mail transfer agent; maintaining a confidence factor for said trust score, said confidence factor indicative of the level of confidence for said trust score; maintaining a recency factor for said trust score, said recency factor based on how recently said trust score was updated; and generating a first trust recommendation for said first remote mail transfer agent, said trust recommendation based on a function of said trust score, said confidence factor and said recency factor.

5. A method as claimed in claim 4, wherein said step of forwarding comprises forwarding said first trust recommendation from said first mail transfer agent to a respective trust manager associated with said at least one collaborating mail transfer agent.

6. A method as claimed in claim 4, wherein said mail filter comprises a threshold to determine how an email received is to be processed, and wherein said step of adjusting comprises scaling said threshold in accordance with said trust score.

7. A method as claimed in claim 1 wherein said first and second remote mail transfer agents are either the same or different mail transfer agents.

8. A computer program product stored on a computer readable medium comprising a set of instructions which, when executed on a computer, are operable to implement the steps of the method of claim 1.

9. A trust manager operable on a first mail transfer agent having a mail filter arranged to receive at least one e-mail sent from a first remote mail transfer agent, said manager comprising;

(a) means, responsive to said mail filter filtering at least one email received from said first remote mail transfer agent, for generating a first recommendation for said first remote mail transfer agent based on said filtering; (b) means for forwarding said first recommendation from said trust manager to at least one collaborating mail transfer agent;

(c) means for receiving from a collaborating mail transfer agent at least one recommendation in relation to a second remote mail transfer agent; and

(d) means for adjusting said mail filter in relation to emails received from said second remote mail transfer agent based on said at least one received recommendation.

10. A mail transfer agent comprising a trust manager according to claim 9 and a mail filter for filtering said at least one email on said first mail transfer agent.