GB2450897A

GB2450897A - Identifying network hosts running on a computer network

Info

Publication number: GB2450897A
Application number: GB0713414A
Authority: GB
Inventors: William Harris
Original assignee: Tideway Systems Ltd
Current assignee: Tideway Systems Ltd
Priority date: 2007-07-11
Filing date: 2007-07-11
Publication date: 2009-01-14
Anticipated expiration: 2027-07-11
Also published as: GB2450897B; GB0713414D0

Abstract

An automated method, system and appliance for enumerating and mapping, the underlying network hosts 2a, 2b, 3 in a particular range of network addresses, and establishing the relationship between hosts and IP addresses including both virtual 3 and physical 2a,2b hosts by analysing responses to attempted or successful network application layer handshakes. The method comprises initiating an application layer handshake for a network address to be identified, then using some or all of the response to the handshake to associate the network address with a specific host. Specific hosts may be identified by reference to distinguishing marks present in, for example, a returned SSH host key, SSL server certificate, unusual TELNET or FTP banner, or HTTP error data generated in response to a request for a document path that is unlikely to exist. Hashing functions may be used to identify distinguishing marks in the handshake responses.

Description

1 2450897 IDENTIFYING NETWORK HOSTS RuNNING ON A COMPUTER

NETWORK

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to a method of identifying network hosts running on a computer network. An implementation of the invention solves the problcm of cnumcrating, using 1() an automated network discovery system, the underlying network hosts in a particular range of network addresses, identifying hosts that support or can be reached at more than one network address, and detecting the relationship, known as mapping, between network addresses on a network and the various hosts running on that network.

A host' is normally understood to be one running instance of a network operating system or software environment. The method described here does not attempt to detect when a number of such instances share the same physical hardware implementation, for example in a virtualized environment.

2. Description of the Prior Art

Reliable identification of the network hosts behind a range of network address is of great value when carrying out projects on computer networks that have many network hosts or when identifying and solving problems on such a network. For example, the answer to the question, "which hosts in this subnet are running this version of this operating system?" is valuable when that version of that operating system is about to become unsupported b the operating system supplier and those hosts need to be retired.

Remote operating system fingerprinting tools such as nmap (described on the website can be used to detect which network addresses are served b' a particular version of an operating system version, but since the underlying hosts will typically have more than one network address, the data is less valuable because it does not provide address to host identity mapping. Further complicating matters, virtual hosts may also be running on physical hosts connected to the network. An abbreviated and simplified diagram of such a network is shown in Fig. 1, where the network cloud I desknates the network, there are two physical hosts 2a and 2b, physical host 2a in turn has a virtual host 3 running on it and the hosts 2a, 2b, and 3 have network addresses 4a, 4b, 4c, 4d and 4e. Ncrwork address-to-host mapping consists in identifying which network addresses 4a to 4e are in fact present on or mapped to' a specific host, i.e., 2a, 2b or 3. Moreover, network address-to-host identity mapping data is often not readily available in a reliable firm in organisations undertaking such projects. This may be due, for example, to a legacy of uncontrolled changes, an accumulation of errors in a manual process, or an inability of a process to keep pace with the rate of change.

Approaches and methods already exist to solving this problem in the prior art. These approaches and methods fall into two general categories: (i) Direct cooperation with software on the host; or (ii) Observation of low-level behaviours of the hosts' TCP/IP stacks Direct Co-operation with Software on the Host This first described prior art approach relies on having the abihtv to gain access to the host. This access is conventionally obtained with a software agent, a small piece of software provided by the vendor of the automated network discovery system and installed on the hosts that are to be scanned. Alternatively, access is gained via the indigenous remote access methods, such as SSFI login or \VI\II, that are already provided by the hosts' operating systems. In this approach, some sort of trust relationship must be built between the automated network discovery system and the hosts being accessed.

This can be accomplished by giving an appropriate remote login credential to the discovery system.

Once access to the host is obtained, it is usually trivial to find which other network addresses the host has. For example, once an appropriate level of access has been gained to a GNU/Linux system shell, the list of network interfaces known to the operating s'stem and their associated addresses can be found b parsing the output of the ifconfig command. Alternately, if the discovery system has a sufficient level of access, it can create a persistent file containing a unique identifier that can be examined at any subsequent login session with that credential.

If the discovery system uses an installed software agent, the agent can query the operating system for its list of network addresses. Also, the agent itself can record, or be installed with, a unique identifier that can be queried b' the discovery system.

Ama;or disadvantage of approaches in this category is that changes must be made to the host (to install an agent or credential) before it can he remotely identified. To make such changes to a large number of hosts across a large IT environment is costly and, crucially, requires that each host first be manually identified befire this technique allows the process to be automated.

Observation of Low-Level Behaviours of the 1-losts' TCP/IP stacks These approaches seek to identify hosts by distinguishing between instances of TCP/IP stacks. They rely on the observation that, even if two TCP/IP stacks have identical software, their behaviour is also determined by an internal state that is likely to be different for each running instance of the software. Some parts of this state have an influence on the external behaviour of the stack that allows a remote observer to detect when a set of network addresses are all influenced by the same state (or a set of states with the same recent ancestor, i.e., that originated in a single common state.) One such approach, (described in a paper by Steven M. Bellovin, entitled -4 Techmqiirfir Counting N-lTied Hosts, published by AT&T Labs Research in November 2002 (available from http://www.cs.columbia.edu/ -smb/papers/ fnat.pd f) is based on the observation that the 16-bit id' field in IP packet headers (see Request for Comment 791 (available at bup://w'ww.Liqs.or2!rics/r1c9l.html)) is usually implemented by the TCP/IP stack as a simple wrap-around counter, or as a byte swapped' wrap-around counter, in which the most significant and least significant bytes have been swapped. Over time, differences in outgoing network traffic will cause the counters in otherwise identical hosts to drift apart.

The IPid field is primarily intended to be used in distinguishing fragment packets of one IP datagram from those of another. There is no requirement that the field be implemented as a counter, only that it must be "unique for that source-destination pair and protocol for the time that datagram is active in the internet system." However, presumably because counters are a vet-v simple method of generating series of unique numbers, many TCP/IP implementations use them. Analysing the IPid fields from packets received from a set of network addresses reveals the underlying hosts as patterns of similarly increasing counters.

A disadvantage of this approach is that there are only 65536 possible states for the IPid field, making it possible that results from two or more separate hosts will coincide. This is not a significant problem when the number of hosts is small enough that the patterns can be resolved despite these events, but as the number of hosts increases, the patterns could become impossible to detect. Another disad antage is that not ever TCP/IP stack uses a simple counter for the IPid fieki. Some operating svstenis use a pseudo-random number generator, making the analysis much more difficult.

Another such approach (described by See T. kohno, A. Broido, and K. Claffv in Remote p/Jyüca/ dezice jiierin/ii published in IEEE Transactions on Dependable and Secure Computing, 2(2), 2005 and available from bnp:/ !w'w.cse.ucse.edu/usersi tk)hfu.papcrsiPDF.'KoBrO200pd Extended -lowres.pd is based on the observation that a distinct host will usually be running on a distinct physical computer with a unique physical system clock. Over time, microscopic differences in physical system clocks result in a small, but measurable, differences in the times reported by those clocks, called clock skew. The clock skew can be revealed in the response to an 1CMP Timestamp Request (see Request for Comment 792 (available at htq:/ /wv.faqs.org/rfcs/rfc792.htm!)) is or from the TCP timestamp option (see Request for Comment 1323 (available at is.

Every network address served b a particular host shares the same unique clock skew, which can be used to identify the host.

One disadvantage of this approach is that the clock skew is a property of the physical computer on which the operating system is running. This technique cannot distinguish between separate operating system instances that are sharing one physical computer, such as in virtualized environment, which of necessity would share the same clock skew.

SUMMARY OF THE PRESENT INVENTION

In a first aspect of the present invention, there is a method of scanning a network of computers to determine the mapping between network addresses and the underlying operating system instances. This is accomplished b initiating an application-layer handshake with each network address, and then using one or more parts of the responses to determine hen one or more network addresses belong to the same hosts (i.e., operating system instance or software environment.) Methods for triering potentially identifying responses from the handshakcs of the SSII (i.e., Secure Shell), SSJ (Secure Socket Laver, TLS (Transport Layer Security), TELNET, FTP (File Transfer), and l-l'TTP protocols are described, as well as methods for extracting the identifying features of those responses, sometimes referred to herein as patterns. Triering a response does not necessarily require that the attempt to establish communications by application layer handshake be successful, merely that it be enough to tner a response from the network address from which identifying features specific to the host, can be identified.

The present invention is predicated on the counter-intuitive insight that network hosts can be made to divulge reliable identiaing information about themselves without the need to provide access credentials or to alter the host. Prior approaches made use of unreliable mechanisms or required that the hosts being identified be altered in some way.

Instead, the present invention makes use of the identifying features of the data sent by the host as part of the normal handshaking stages of application-layer network protocols.

Handshaking' is a term of art used to refer to predetermined hardware or software activity or protocol designed to establish or maintain communication between two systems or programs. In its most basic form a handshake is a request from one system or network address to another to establish communications using a specific protocol or format, which the requested network address responds either permitting the communication or refusing it (i.e., it fails.)

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 is a representation of a generic network 1, that includes two physical hosts 2a and 2b and a Virtual Host 3 exIsting on Physical Host 2a. Each of the hosts has network addresses 4a, 4b, 4c, 4d and 4e.

Figure 2 is a representation of an embodiment of thee invention wherein a network appliance consists of a Scanner 5 and a Darastore 6, and where Scanner 6 sends 1-Iandshake Requests 7 eliciting I-Iandshakc Response 8.

Figure 3 is a graphical representation of 5 handshake responses. Two responses share a common distinguishing mark graphically represented as pattern Wa, indicating that the responses originated from a common host, notwithstanding the different network addresses ha and lib of the responses. One response has a unique distinguishing mark graphically represented as pattern lob, associating that distingusing mark with a sole network address 11 c. Two additional responses share a second common distinguishing mark lOc, notwithstanding different network addresses lid and lie, indicating that these two network addresses share a common host. All of the distinguishing marks bear time stamps i2a, 12b, 12c, 12d and 12e.

Figure 4 shows the relationship between the Scanning Engine 5 and Datastore 6. The Scanning Engine is shown as comprising two components: a Network Address Queue; and a store or database of corresponding handshake responses for those network addresses for which a Handshake has been attempted (although it could simply contain index entries to the data store allowing it to locate the relevant handshake responses.) The Scanning Engine 5 in turn compares the Network Addresses with the host information including distinguishing marks stored in the Datastore in correspondence with IP addresses identified for each host. The Datastore is represented as containing records for Hosts 2a, 2b and 3 represented in Figures 1 and 2.

Figure 5 shows the process b which a handshake response for a previously unidentified host is stored to the Datastore, SO that the time stamp 12d is discarded and the distinguishing mark lOc is stored in a record corresponding to the Network Address lid.

DETAILED DESCRIPTION

One embodiment of the invention allows a set of IP addresses to be grouped into hosts, based on how the respond to Secure Shell (commonly known as SSH') protocol connection attempt as described in Request for Comment 4251 (available at hi Lp:/ . v'v.Iaqs.org/rfcsrfc42 I.lmnl). The system comprises the following: i. A datastorc'; and ii. A scanning engine.'

Description of the datastore'

The datastore is typically a software device running on a computer system, and can be database stored in volatile or non-volatile memory or a program that associates items of information with one another. It provides a software interface to a database capable of at least storrng and retrieving a number of host records. Each host record has at least the followIng data: (i) a set of IP addresses, possibly empty; and (ii) a host key.

The software interface provided by the datastore provides access to at least the queries and operations described below. The datastore supports at least the following queries: a. supply the set of host records whose set of IP addresses contains the supplied IP address.

b. supply the set of host records that have an empty set of IP addresses.

c. supply the set of host records whose host key is the same as the one supplied.

d. supply the host key and set of IP addresses for every host record.

The datastore supports at least the following operations on the set of host records: * create a new record having the host key and IP address set populated with, respectively, a supplied host kc and a set containing onh a supplied IP address; * delete a specified record.

The datastore supports at least the following operations on host records: * remove the supplied IP address from the set of IP addresses on the record.

* ensure that the set of IP addresses contains a supplied IP address.

Description of the scanning engine'

The scanning engine is typically a software device running in a computer system. It has access to the software interface of the datastore described above and to the network being scanned. It normally maintains a collection of IP addresses to scan, typically in the form of a queue. it provides its own softwarc interface that supports at least the following operations: * add one or more IP addresses to the queue; * start scanning engine running; * stop scanning engine; * interrogate the queue to see if there are no more IP addresses to scan; * interrogate the engine to see if there are no IP addresses being scanned currently.

It is also possible to construct a scanning engine that rather than maintaining a queue of IP addresses to scan, uses a priority queue, a "bag," an unordered set, a single register, or any other container that can be used to maintain a record of the intent to process zero or more network addresses.

Typically, a user of the interface will add some IP addresses to the collection, usually a queue, then start the scanning engine. When it is running, the scanning engine pulls each IP address off the queue in turn and processes them. To process an IP address, the engine normally attempts the following series of steps: * Check to see that there is a host listening for SSH (port 22) at that IP address; * Fail if no TCP connection can be made; * Do the minimum handshaking with the 551-I service that is necessary to obtain the host key; * Search the datastore for an existing record with the same host key; * If no record can be found with that host key, create a new record populated with the host key and a set containing only the current IP address; * If a record with that host key can be found, ensure that its set of IP addresses contains the current IP address.

* Search for records that have the current IP address but that have a different host key; * Remove the current IP address from each of these records; * Search for records that have no IP addresses and remove them.

\Vhen this series of steps has been attempted for the 1P address, the scanning engine pulls the next IP address from the queue and processes that. If there are no more IP addresses in the queue, thc scan is finished. A user can then use the software interfitce to the datastore to query for all the hosts' sets of IP addresses, thereby obtaining the groups of IP addresses that belong to the same host. The first two steps listed above, i.e., checking is a host is listening for SSII connections (i.e., on port 22) and failing if rio TCP connection can be made are not strictl necessary for a successful implementation of the invention, but the inventor regards them as a better implementatk)n of his invention.

In another example, instead of the SSH host key, the system stores and uses the server certificate obtained from an SSL service (as described in Request for Comment 2246 (available at li;L JQrgLt&L ti.1hn1) for example an HTTPS server (as described in Request for Comment 2818 (available at btrp://v.faqs.urg/rfcs!rfc2818.htnil) or alternately the certificate from a Transport Layer Security service as described in Request for Comment 4346 (available at lt://v*-'.fac1s.4)rg rfcs! rfc4M6.html.

In another example, the system uses the I'ELNET banner' -that is, the data immediately disclosed to the client when connecting to a service at the IP address using the TELNET protocol.

Since TELNET banners are often standardized and do not contain host identification information, in this example, the system filters the TELNET banner responses, and only considers those that are unusual to indicate that subsequently visited IP addresses belong to the same host as a prior IP address.

In this example, the system has database of patterns used to distinguish between banners that contain host identification information and banners that do not contains host identification information. The database allows a variety of patterns to be described.

For example, a pattern can match banners that are shorter than a certain length, or banners matching a regular expression.

In another example, the system uses similar banners provided by an FTP (Pile Transfer Protocol) service (as described in Request for Comment 959 (available at http:/ /wvwfaqs.org/rfcs/rfc959.hun). Again, the system ignores common banners using a similar database of patterns to the one used in the previous (telnet) example.

In another example, the system uses an l-ITTP (1 lvper Text Transfer Protocol) request for the root document (also described in Request for Comment 959. Again it ignores non-identifying responses using a database of patterns.

In another example, the system uses and l-ITTP request for a document at path that is unlikely to exist. Such a path is formed from a globally unique identifier using a random number generator in the scanning system. The resultant error response (HTTP error 404) will contain possibly identifying information.

In another example, the system uses a database of patterns to extract only a part of a response, or suppress parts of the response that change frequently but that do not contribute to identification. For example, a mainframe that responds to TELNET (as described in Request for Comment 854 (available at htq://vnv.faqsorg/rfcs!rfc854.html)) with a banner that contains a time or date would appear different whenever the date or time changed. By adding a pattern that extracts the useful part of the banner, suppressing the frequently changing parts, the system can again make sense of the response.

In this example, the system also has knowledge of patterns that it should always regard as non-useful. For example, substrings of the response that match the group in the regular expression (meaning three pairs of digits separated b' colons, See htr4llrLdmkzJ4zctnQJi1bns2LLa2c3 look like timestamps and should be ignored.

Depending on the circumstance, the system will either use one or more parts of the response directly as identifiers, for example a hostname in the banner, or it will use the result of some calculation on the data, a hash function' for example.

An example of applying a hash function is as fi)llows. Suppose the approach used is the tactic of sending a 1-1TTP request for a document at a path that is unlikely to exist, at IP addresses 192.168.0.9 and 10.0.2.1 where both return the same response when the same unlikely URL is requested: <!DOCTYPE 1-ITMI. PUBLiC "-//IETF//DTD HTML 2.0//EN"> <HTML><HEAD> <TITLE>404 Not Found</TITLE> <I HEAD><BODY> <H1>Not Found</H1> The requested URL /iexpecta404 was not found on this server.<P> <HR> <ADDRESS>Apache/1.3.33 Server at fawkesiocal Port 80</ADDRESS> </BODY> </HTML> In this case, one possible distinguishing mark is "Apache/1.3.33 Server at fawkes.local Port 80," which could be extracted using a regular expression like "(?s)<ADDRESS>(["<]+)</ADDRESS." Alternately a hash function is applied to the extracted text and this creates an expression, which though not typically reversible, if used consistently can be stored as a derived identifying pattern. Thus, by way of example, if the MD5 algorithm (described in Request for Comment 1321 at Imp:t 321.hml is applied to the extracted text this yields the gives the 128-bit value 251770196952399805744)676047310317281 0( )0, which then can be used in the Datastorc as a pattern for identifying the hosts, even though in principle it does not contain sufficient information to reconstruct the original response is derived from that response in such a way as to be distinctive and usable to identify other instances where the same host issues a response.

These example sources of distinguishing marks are not exhaustive. An embodiment can uses number of techniques, including combinations of the foregoing, for extracting distinguishing marks and combines them to create a grouping of IP addresses into hosts.

As well as producing a grouping with a high confidence, it would also broaden the coverage of hosts that can be identified.

The invention can be embodied in a number of ways, as a method, or part of a larger system, or a1ternatc1' as a network appliance, i.e., an item of hardware designed to carry out various processes on a computer network.

Claims

1. A method of identifying hosts running on a computer network in which more than one physical or virtual host is present, comprising the steps of: * Initiating an application layer handshake for a network address to be identified; * Using some of all of the response to the application layer handshake to associate the network address with a specific host.

2. The method of Claim 1, including the step of: * Finding in the response to the application layer handshake, a distinguishing mark characteristic of the host for such network address; * Associating the network address with a specific host b means of the distinguishing mark.

3. The method of Claim I where the application layer handshake comprises attempting to establish a Secure Shell connection with the network address to be identified.

4. The method of Claim 2 where the distinguishing mark found for the application layer handshake is a part or all of a host key.

5. The method of Claim 2 where the distinguishing mark, or a value derived from the distinguishing mark, is stored in a datastore in ConjunCtion with a network address or addresses identified as corresponding to the host with which the distinguishing mark is associated.

6. The method of Claim 3 where the distinguishing mark is stored in a datastore in conjunction with a network address or addresses identified as corresponding to the host with which the distinguishing mark is associated.

7. The method of Claim 4 where the host key, or a value derived from the host key, is stored in a datastore in conjunction with network address or addresses identified as corresponding to the host with which the distinguishing mark is associated.

8. The method of Claim 1 where the application layer handshake comprises Secure Socket Liver requests sufficient to causc the return of a server certificate from the network address to be identified.

9. The method of Claim 2 where the distinguishing mark found for the application layer handshake is a part or all of a server certificate returned in response to Secure Socket Layer request.

10. The method of Claim 9 where the server certificate, or a value derived from the server certificate, is stored in a datastore in conjunction with network address or addresses identified as corresponding to the host with which the server certificate is associated.

11. The method of Claim I where the application layer handshake comprises Transport Layer Security requests sufficient to cause the return of a server certificate from the network address to be identified.

12. The method of Claim 2 where the distinguishing mark found for the application layer handshake is a part or all of a server certificate returned in response to Transport Layer Security request.

13. The method of Claim 12 where the server certificate, or a value derived from the server certificate, is stored in a datastore in conjunction with network address or addresses identified as corresponding to the host with which server certificate is associated.

14. The method of Claim 1 where the application layer handshake comprises an attempt to connect to the network address to be identified using the TELNET protocol.

15. The method of Claim 2 where the distinguishing mark identified for the application layer handshake is a part or all of a TELNET banner returned in response to an attempt to connect to the network address to be identified using the TELNET protocol.

16. The method of claim 15 where some or all of the TELNET banner, or a value derived from the TELNET banner, is stored in a datastore in con junction with network address or addresses identified as corresponding to the host with which tile TELNET banner mark is associated.

17. The method of claim 16 where a datastore is used to distinguish between those TELNET banners that contain host identifying information and those TELNET banners that do not.

18. The Method of Claim I where the application layer handshake comprises an attempt to connect to network address to be identified using a File Transfer Protocol.

19. The method of Claim 2 where the distinguishing mark identified frr the application layer handshake is a part or all of a File Transfer Protocol banner returned in response to an attempt to connect to the network address to be identified using a File Transfer Protocol

20. The method of Claim 19 where some or all of the File Transfer Protocol banner, or a value derived from the File Transfr Protocol banner, is stored in a datastore in conjunction with network address or addresses identified as corresponding to the host with which the File Transfer Protocol banner is associated.

21. The method of claim 20 where a datastore is used to distinguish between those File Transfer Protocol banners that contain host identifying information and those File Transfer Protocol banners that do not.

22. The Method of Claim 1 where the application layer handshake comprises a Hypertext Transfer Protocol request, to the network address to be identified, for a root document.

23. The method of Claim 2 where the distinguishing mark identified for the application layer handshake is a part or all of a root document returned in response to a I ivpertext Transfer Protocol request.

24. Thc method of Claim 23 where some or all of the root document, or a value derived from the root document, is stored in a datastore in conjunction with network address or addresses identified as corresponding to the host with which the root document is associated.

25. The method of claim 23 where a datastore is used to distinguish between those root documents that contain host identifying information and those root documents that do not.

26. The Method of Claim I where the application Liver handshake comprises a Hypertext Transfer Protocol request request to the network address to be identified for a document at a path that is unlikely to exist.

27. The method of Claim 2 where the distinguishing mark identified for the application layer handshake is part or all of an error response returned in response to a Hypertext Transfer Protocol request request to the network address to be identified for a document at a path that is unlikely to exist.

28. The method of Claim 27 where some or all of the error response, or a value derived from the error response, is stored in a datastore in Conjunction with network address or addresses identified as corresponding to the host with which the error response is associated.

29. The method of Claim 28 where a datastore is used to distinguish between those error responses that contain host identifying information and those error responses that do not.

30. The Method of Claim I where the application layer handshake comprises a request for secure encrypted communication sufficient to cause the return of an encryption ke or encryption certificate from the network address to be identified.

31. The method of Claim 2 where the distinguishing mark found for the application layer handshake is a part or all of an encryption key or encryption certificate returned in response to a request for secure encrypted communication.

32. The method of Claim 31 here some or all of the encryption key or encryption certificate, or a value derived from the encryption key or encryption certificate, is stored in a datastore in conjunction with network address or addresses identified as corresponding to the host with which the encryption key or encryption certificate is associated.

33. The method of Claim 1 in which multiple application layer handshakes are executed for one or more network addresses and the application layer handshakes consist of ant' combination of the handshakes described in the foregoing claims.

34. The method of any of the foregoing claims, wherein frequently changing parts of responses to application layer handshake requests are suppressed or subtracted from said responses.

35. The method of claim 34 wherein the frequently changing part of responses suppressed or subtracted is a time stamp or time record.

36. The methods of claim 33, 34 or 35 where a datastore is used to store responses or distinguishing marks, or values derived from such responses or distinguishing marks, resulting from more than one type of application layer handshake and associates different types of application layer handshake responses or distinguishing marks with a common host.

37. The method of any of the foregoing claims wherein a scanning engine with access to the datastore and the network is used to initiate application layer handshakes

38. The method of an' of the foregoing claims where the wherein the scanning engine compares said handshake response with information contained in the datastore and modifies the record in the datastore in accordance with the handshake response.

39. The method of Claim 38 wherein the scanning engine uses an automated process for associating network addresses with their hosts on a network, that comprises the fi)llowing steps: * Establishing a collection of network addresses to be scanned; * 1or each network address checking if a host is listening at that address; * Carrying out the minimum handshake attempt fir a each given netvork address necessary to obtain a host response containing a distinguishing i-nark; * Searching the datastore to determine if it has records of a host with the distinguishing mark provided in the aforementioned host response; * If no record is found with the aforementioned distinguishing mark, creating a new host record with the given network address; * If a record is found with the aforementioned distinguishing mark, amending that record to reflect its relationship with the given network address.

40. A system for identifying hosts running on a computer network in which more than one physical or virtual host is present, identifying the network address or network addresses associated with a host and establishing the relationship between identified network addresses and identified hosts, characterised in that the system: * Initiates an application layer handshake with each network address to be identified; * Uses some or all of the response to the application layer handshake request to associate a nervork address or network addresses with a specific host.

41. The system of Claim 40, further characterised in that it: * Identifies from the response to the application layer handshake request a distinguishing mark characteristic of the host for a network address or addresses; * Compares the distinguishing mark identified for each such response to an application layer handshake with at least a second response to an application lawyer handshake; * Determines from the comparison of the responses to handshake requests whether one or more network addresses are associated with a common host.

42. The system of Claim 4() where the handshake comprises attempting to establish a Secure Shell connection with the network address to be identified.

43. The system of Claim 41 where the distinguishing mark identified for the application layer handshake is a part or all of a host key returned in response to an attempt to establish a Secure Shell connection.

44. The system of Claim 41 where the distinguishing mark, or a value derived from the distinguishing mark, is stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the distinguishing mark is associated.

45. The system of Claim 4() where the handshake comprises Secure Socket Layer requests sufficient to cause the return of a server certificate from the network address to be identified.

46. The system of Claim 41 where the distinguishing mark identified for the application layer handshake is a part or all of a server certificate returned in response to a of Secure Socket Layer request.

47. The System of Claim 45 where some or all of the server certificate, or a value derived from the server certificate, is stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the some or all of the server certificate is associated.

48. The System of Claim 46 where some or all of the distinguishing mark is stored in a datastore, or a value derived from the distinguishing mark, in conlunction with network addresses identified as corresponding to the host with which the some or all of the distinguishing mark is associated.

49. The system of Claim 40 where the comprises Transport Layer Security requests sufficient to cause the return of a server certificate from the network address to be identified.

50. The system of Claim 49 where the distinguishing mark identified for the application layer handshake is a part or all of a server certificate returned in response to a Transport Layer Security reucst.

51. The System of Claim 49 where some or all of the server certificate, or a value derived from the server certificate, is stored in a datastore in conuncnon with network addresses identified as corresponding to the host with which the some or all of the server certificate is associated.

52. The System of Claim 50 where some or all of the distinguishing mark is stored in a datastorc in conjunction w-itb network addresses identified as corresponding to the host with which the some or all of the distinguishing mark is associated.

53. The system of Claim 40 where the handshake comprises an attempt to connect to a network address to be identified using the TELNET protocol.

54. The system of Claim 41 where the distinguishing mark identified for the application layer handshake is a part or all of a TELNET banner returned in response to an attempt to connect to a network address.

55. The system of Claim 43 where a datastore of whole or complete TELNET banner patterns is used to distinguish between those TELNET banners that contain host identifying information and those TELNET banners that do not.

56. The System of Claim 54 where distinguishing marks derived from TELNET banners, are stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the each TELNET banner is associated.

57. The system of Claim 40 where the handshake comprises an attempt to connect to network address to be identified using a File Transfer Protocol.

58. The system of Claim 41 where the distinguishing mark identified for the application layer handshake is a part or all of a File Transfer Protocol banner returned in response to an attempt to connect to network address.

59. The system of claim 50 where a datastore of whole or complete File Transfer Protocol banner patterns, or a values derived from File Transfer Protocol banner patterns, is used to distinguish between those File Transfer Protocol banners that contain host identifying information and those File Transfer Protocol banners that do not.

60. The system of Claim 58 or 59 where distinguishing marks derived from File Transfer Protocol banners are stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the distinguishing mark is associated.

61. The system of Claim 40 where the handshake comprises a Hypertext Transfer Protocol request to the network address to be identified for a root document.

62. The system of Claim 41 where the distinguishing mark identified for the application layer handshake is a part or all of a root document returned in response to a I-lvpertext Transfer Protocol request.

63. The system of Claim 61 where a datastore of root documents, or values derived from root documents, is used to distinguish between those root documents that contain host identifying information and those root documents that do not.

64. The system of Claim 62 where a datastore of distinguishing marks is used to distinguish between those root documents that contain distinguishing marks and those root documents that do not.

65. The System of Claim 61 where some or all of a root document returned in response to a I lvpertext Transfer Protocol request, or a value derived from such as root document, is stored in a datastore in conlunction with root documents and any network addresses identified as corresponding to the host with which the distinguishing mark is associated.

66. The System of Claim 62 where distinguishing marks derived from File Transfer Protocol banners are stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the distinguishing mark is associated.

67. The system of Claim 40 where the handshake comprises a I Ivpcrtext Transfer Protocol request request to the network address to be identified for a document at a path that is unlikely to exist.

68. The system of Claim 41 where the distinguishing mark identified for the application layer handshake is some or all of an error response to a Hypertext Transfer Protocol request.

69. The system of Claim 67 where a datastorc of error responses is uscd to distinguish between those error responses that contain host identifying information and those error responses that do not.

70. The system of Claim 67 where a datastore of distinguishing marks is used to distinguish between those error responses that contain host identifying information and those error responses that do not.

71. The system of Claim 41) where the application layer handshake comprises a request for secure encrypted communication sufficient to cause the return of an encryption key or encryption certificate from the network address to be identified.

72. The system of Claim 41 where the distinguishing mark found for the application layer handshake is a part or all of an encryption key or encryption certificate returned in response to a request for secure encrypted communication.

73. The system of an' of the preceding Claims 40 to 72 wherein frequently changing parts of responses to handshake requests are suppressed or subtracted from said responses.

74. The system of Claims 40 to 72 in which handshakes to one or more network addresses consists of any combination of the handshakes described in Claims 42 to 72.

75. The system of claim 74 where a datastore is used to store responses or distinguishing marks resulting from more than one type of application laser handshake, or values derived from such application laer handshakes, and associate different types of application layer handshake responses or distinguishing marks with a common host.

76. The system of an' of the foregoing Claims 4() to 75 wherein a scanning engine with access to the datastore and the network is used to initiate application layer handshakes with multiple network addresses.

77. The system of Claim 76 whcrein the scanning engine is adapted to perform an automated OCCS5 for associating network addresses with their hosts on a network, that comprises the following steps: * Establishing a collection of network addresses to be scanned; * For each network address checking if a host is listening at that address; * Carrying out the minimum handshake attempt for a each given network address necessary to obtain a host response containing a distinguishing mark; * Searching the datastore to determine if it has records of a host with the distinguishing mark provided in the aforementioned host response; * If no record is found with the aforementioned distinguishing mark, creating a new host record with the given network address; * If a record is found with the aforementioned distinguishing mark, amending that record to reflect its relationship with the given network address.

78. An appliance connected to a computer network for identifying hosts running on the computer network in which more than one physical or virtual host is present, identifying the network address or network addresses associated with a host and establishing the relationship between identified network addresses and identified hosts, characterised in that the appliance: * Initiates an application layer handshake with each network address to be identified; * Uses some or all of the response to said application layer handshake request to associate a network address or network addresses with a specific host.

79. The appliance of Claim 78, further characterised in that it: * identifies from the response to said application layer handshake request a distinguishing mark characteristic of the host for a network address or addresses; * Compares the distinguishing mark identified for each such response to an application layer handshake with at least a second response to an application lawyer handshake; * Determines from the comparison of the responses to handshake requests whether one or more network addresses are associated with a common host.

80. The appliance of Claim 78 here the handshake comprises attempting to establish a Secure Shell connection with the network address to be identified.

81. The appliance of Claim 79 where the distinguishing mark identified for the application layer handshake is a part or all of a host key returned in response to an attempt to establish a Secure Shell connection.

82. The appliance of Claim 79 where the distinguishing mark, or a value derived from the distinguishing mark, is stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the distinguishing mark is associated.

83. The appliance of Claim 78 where the handshake comprises Secure Socket Layer requests sufficient to cause the return of a server certificate from the network address to be identified.

84. The appliance of Claim 79 where the distinguishing mark identified for the application layer handshake is a part or all of a server certificate returned in response to a of Secure Socket Layer request.

85. The appliance of Claim 83 where some or all of the server certificate, or a value derived from the server certificate, is stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the some or all of the server certificate is associated.

86. The Appliance of Claim 84 where some or all of the distinguishing mark is stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the some or all of the distinguishing mark is associated.

87. The appliance of Claim 78 where the handshake comprises Transport Layer Security requests sufficient to cause the return of a server certificate from the network address to be identified.

88. The appliance of Claim 79 where the distinguishing mark identified for the application la cr handshake is a part or all of a server certificate returned in CSOflSC to a Transport Layer Security request.

89. The Appliance of Claim 87 where some or all of the server certificate, or a value derived from the server certificate, is stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the some or all of the server certificate is associated.

90. The Appliance of Claim 88 where some or all of the distinguishing mark is stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the some or all of the distinguishing mark is associated.

91. The appliance of Claim 78 where the handshake comprises an attempt to connect to a network address to be identified using the TELNET protocol.

92. The appliance of Claim 79 where the distinguishing mark identified for the application layer handshake is a part or all of a TELNET banner returned in response to an attempt to connect to a network address.

93. The appliance of Claim 91 where a datastore of whole or complete TELNET banner patterns, or values derived from TELNET banners, is used to distinguish between those TELNET banners returned in an attempt to connect to a network address that contain host idenrifring information and those TELNET banners that do not.

94. The Appliance of Claim 92 where distinguishing marks derived from TELNET banners are stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the each TELNET banner is associated.

95. The appliance of Claim 78 where the handshake comprises an attempt to connect to net-work address to be identified using a File Transfer Protocol.

96. The appliance of Claim 79 where the distinguishing mark identified for the application layer handshake is a part or all of a File Transfer Protocol banner returned in response to an attempt to connect to network address.

97. The appliance of claim 95 where a datastore of whole or complete File Transfer Protocol banner patterns, or values derived from File Transfer Protocol banner patterns, is used to distinguish between those File Transfer Protocol banners that contain host identifying information and those File Transfer Protocol banners that do not.

98. The Appliance of Claim 96 or 97 where distinguishing marks derived from File Transfer Protocol banners are stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the distinguishing mark is associated.

99. The appliance of Claim 78 where the handshake comprises a Hypertext Transfer Protocol request to the network address to be identified for a root document.

100. The appliance of Claim 79 where the distinguishing mark identified for the application layer handshake is a part or all of a root document returned in response to a Hypertext Transfer Protocol request.

101. The appliance of Claim 99 where a datastore of root documents, or values derived from root documents, is used to distinguish between those root documents that contain host identifying information and those root documents that do not.

102. The appliance of Claim 100 where a datastore of distinguishing marks is used to distinguish between those root documents that contain distinguishing marks and those root documents that do not.

103. The appliance of Claim 99 where some or all of a root document returned in response to a Hypertext Transfer Protocol request, or a value derived from such a root document, is stored in a datastore in Conjunction with an' network addresses identified as corresponding to the host with which the root document is associated.

104. The appliance of Claim 100 where distinguishing marks derived from File Transfer Protocol banners are stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the distinguishing mark is associated.

105. The appliance of Claim 78 where the handshake comprises a Hypertext Transfer Protocol request request to the network address to be identified for a document at a path that is unlikely tO exist.

106. The appliance of Claim 79 where the distinguishing mark identified fir the application layer handshake is some or all of an error response to a Hypertext Transfer Protocol request.

107. The appliance of Claim 105 where a datastore of error responses, or values derived from error responses, is used to distinguish between those error responses that contain host identifying information and those error responses that do not.

108. The appliance of Claim 106 where a datastore of distinguishing marks, or values derived from distinguishing marks, is used to distinguish between those error responses that contain host identifying information and those error responses that do not.

109. The system of Claim 78 where the application la'er handshake consists of a request for secure encrypted communication sufficient to cause the return of an encryption key or encryption certificate from the network address to be identified.

110. The system of Claim 79 where the distinguishing mark found for the application layer handshake is a part or all of an encption key or encryption certificate returned in response to a request for secure encrypted communication.

111. The appliance of any of the preceding Claims 78 to 110 wherein frequently changing parts of responses to handshake requests aresuppressed or subtracted from said responses.

112. The appliance of any of the preceding claims 78 to 11) in which handshakes to one or more network addresses consists of any combination of the handshakes described in Claims 80 to 110.

113. The appliance of claim 112 where a datastore is used to store responses or distinguishing marks resulting or derived from more than one type of application layer handshake and associate different types of application layer handshake responses or distinguishing marks with a common host.

114. The appliance of an of the foregoing Claims 78 to 113 wherein a scanning engine with access to the datastore and the network is used to initiate application layer handshakes with multiple network addresses.

115. The appliance of Claim 114 wherein the scanning engine is adapted to perform an automated process for associating network addresses with their hosts on a network, that comprises the following steps: * Establishing a collection of network addresses to be scanned; * For each network address check if a host is listening at that address; * Carrying out the minimum handshake attempt for a each given network address necessary to obtain a host response containing a distinguishing mark; * Searching the datastore to determine if it has records of a host with the distinguishing mark provided in the aforementioned host response; * If no record is found with the aforementioned distinguishing mark, creating a new host record with the given network address; * If a record is found with the aforementioned distinguishing mark, amending that record to reflect its relationship with the given network address.

115. The appliance of Claim 114 wherein the scanning engine is adapted to perform an automated process for associating network addresses with their hosts on a network, that comprises the following steps: * Establishing a collection of network addresses to be scanned; * For each network address check if a host is listening at that address; * Carrying out the minimum handshake attempt for a each given network address necessary to obtain a host response containing a distinguishing mark; * Searching the datastore to determine if it has records of a host with the distinguishing mark provided in the aforementioned host response; * If rio record is found with the aforementioned distinguishing mark, creating a new host record with the given network address; * If a record is found with the aforementioned distinguishing mark, amending that record to reflect its relationship with the given network address.

Amendments to the claims have been filed as follows

CLAIMS -

1. A method of identifying hosts running on a computer network in which more than one physical or virtual host is present, comprising the steps of: Irutiating an application layer handshake for a network address to be identified; * Using some or all of the response to the application layer handshake to associate the network address with a specific host.

2. The method of Claim 1, including the step of: * Finding in the response to the application layer handshake, a distinguishing mark characteristic of the host for such network address; * Associating the network address with a specific host by means of the distinguishing mark.

:.,, 3. The method of Claim I where the application layer handshake comprises *I* *** attempting to establish a Secure Shell connection with the network address to be identified. -: 4. The method of Claim 2 where the distinguishing mark found for the application layer handshake is a part or all of a host key.

* 5. The method of Claim 2 where the distinguishing mark, or a value derived from the distinguishing mark, is stored in a datastore in conjunction with a network address or addresses identified as corresponding to the host with which the distinguishing mark is associated.

7. Thç method of Claim 4 where the host key, or a value derived from the host key, is stored in a datastore in conjunction with network address or addresses identified as corresponding to the host with which the distinguishing mark is associated.

8. The method of Claim 1 where the application layer handshake comprises Secure Socket Layer requests sufficient to cause the return of a server certificate from the network address to be identified.

9. Thc method of Claim 2 where the distinguishing mark found for the application layer handshake is a part or all of a server certificate returned in response to Secure Socket Layer request.

14. The method of Claim I where the application layer handshake comprises an attempt to connect to the network address to be identified using the TELNET protocol.

16. The method of claim 15 where some or all of the TELNET banner, or a value derived from the TELNET banner, is stored in a datastore in conjunction with network address or addresses identified as corresponding to the host with which the TELNET banner mark is associated.

18. The Method of Claim I where the application layer handshake comprises an attempt to connect to network address to he identified using a File Transfer Protocol.

19. The method of Claim 2 where the distinguishing mark identified for the application layer handshake is a part or all of a File Transfer Protocol banner returned in response to an attempt to connect to the network address to be identified using a File Transfer Protocol 20. The method of Claim 19 where some or all of the File Transfer Protocol banner, or a value derived from the File Transfr Protocol banner, is stored in a datastore in Conjunction with network address or addresses identified as corresponding to the host with which the File Transfer Protocol banner is associated.

22. The Method of Claim I where the application layer handshake comprises a Hypertext Transfer Protocol request, to the network address to be identified, for a root document.

23. The method of Claim 2 where the distinguishing mark identified for the application layer handshake is a part or all of a root document returned in response to a Hypertext Transfer Protocol request.

24. The method of Claim 23 where some or all of the root document, or a value derived from the root document, is stored in a datastore in conjunction with network address or addresses identified as corresponding to the host with which the root document is associated.

26. The Method of Claim I where the application layer handshake comprises a I-Ivpertext Transfer Protocol request request to the network address to be identified for a document at a path that is unlikely to exist.

27. The method of Claim 2 where the distinguishing mark identified for the application layer handshake is part or all of an error response returned in response to a l-lvpertext Transfer Protocol request request to the network address to be identified for a document at a path that is unlikely to exist.

29. The method of Claim 28 where a datastore is used to distinguish between those error responses that contain host identifying information and those error responses that do hot.

30. The Method of Claim 1 where the application layer handshake comprises a request fir secure encrypted communication sufficient to cause the return of an encr\'ption key or encption certificate from the network address to be identified.

31. The method of Claim 2 where the distinguishing mark found for the application laser handshake is a part or all of an encryption key or encryption certificate returned in response to a request for secure encrypted communication.

32. The method of Claim 31 where some or all of the encryption key or encryption certificate, or a value derived from the encryption key or encryption certificate, is stored in a datastore in conlunction with network address or addresses identified as corresponding to the host with which the encryption key or encryption certificate is associated.

33. The method of Claim I in which multiple application layer handshakes are executed for OflC or more network addresses and the application layer handshakes consist of any combination of the handshakes described in the foregoing claims.

34. The method of any of the ftregoing claims, wherein frequently changing parts of responses to application layer handshake requests are suppressed or subtracted from said resp)nses.

37. The method of an of the foregoing claims wherein a scanning engine with access to the datastore and the network is used to initiate application laer handshakes 38. The method of an' of the foregoing claims where the wherein the scanning engine compares said handshake response with information contained in the datastore and modifies the record in the datastore in accordance with the handshake response.

39. The method of Claim 38 wherein the scanning engine uses an automated process for associating network addresses with their hosts Ofl a network, that comprises the following steps: * Establishing a collection of network addresses to be scanned; * For each network address checking if a host is listening at that address; * Carrying out the minimum handshake attempt for a each given network address necessary to obtain a host response containing a distinguishing mark; * Searching the datasrore to determine if it has records of a host with the distinguishing mark provided in the aforementioned host response; * If no record is found with the aforementioned distinguishing mark, creating a new host record with the given network address; * If a record is found with the aforementioned distinguishing mark, amending that record to reflect its relationship with the given network address.

40. A system for identifying hosts running on a computer network in which more than one physical or virtual host is present, identifying the network address or network addresses associated with a host and establishing the relationship between identified network addresses and identified hosts, characterised in that the system: * Initiates an application layer handshake with each nenvork address to be identified; * Uses some or all of the response to the application layer handshake request to associate a network address or network addresses with a specific host.

42. The system of Claim 40 where the handshake comprises attempting to establish a Secure Shell connection with the network address to he identified.

45. The system of Claim 40 where the handshake comprises Secure Socket Layer requests sufficient to cause the return of a server certificate from the network address to be identified.

46. The system of Claim 41 where the distinguishing mark identified for the applicath)n layer handshake is a part or all of a server certificate returned in response to a of Secure Socket Layer request.

48. The System of Claim 46 where some or all of the distinguishing mark is stored in a datastore, or a value derived from the distinguishing mark, in conjunction with network addresses identified as corresponding to the host with which the some or all of the distinguishing mark is associated.

49. The system of Claim 40 where the comprises Transport Laser Security requests sufficient to cause the return of a server certificate from the network address to be identified.

50. The system of Claim 49 where the distinguishing mark identified for the application la'er handshake is a part or all of a server certificate returned in response to a Transport Layer Security request.

51. The System of Claim 49 where some or all of the server certificate, or a value derived from the server certificatc, is stored in a datastore in conjunction with flCt\'Ork addresses identified as corresponding to the host with which the some or aLl of the server certificate is associated.

52. The S stern of Claim SI) where some or all of the distinguishing mark is stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the some or all of the distinguishing mark is associated.

53. The system of Claim 41) where the handshake comprises an attempt to connect to a net\ ork address to be identified using the TELNET protocol.

56. The System of Claim 54 where distinguishing marks derived from TELNET banners, arc stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the each TELNET banner is associated.

60. The system of Claim 58 or 59 where distinguishing marks derived from File Transfer Protocol banners are stored in a datastore in con;unction with network addresses identified as corresponding to the host with which the distinguishing mark is associated.

65. The System of Claim 61 where some or all of a root document returned in response to a Hpcrtext Transfer Protocol request, or a value derived from such as root document, is stored in a datastore in conjunction with root documents and any network addresses identified as corresponding to the host with which the distinguishing mark is associated.

66. The Sstem of Claim 62 where distinguishing marks derived from File Transfer Protocol banners are stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the distinguishing mark is associated.

67. The system of Claim 40 where the handshake comprises a I-lvpcrtcxt Transfer Protocol request request to the network address to be identified for a document at a path that is unlikely to exist.

68. The system of Claim 41 where the distinguishing mark identified for the application layer handshake is some or all of an error response to a l-Ivpcrtcxt Transfer Protocol request.

69. The system of Claim 67 where a datastore of error responses is used to distinguish between those error responses that contain host identifying information and those error responses that do not.

70. The s stem of Claim 67 where a datastore of distinguishing marks is used to distinguish between those error responses that contain host identifying information and those error responses that do not.

71. The system of Claim 40 where the application layer handshake comprises a request for secure encrypted communication sufficient to cause the return of an encnption key or encryption certificate from the network address to be identified.

72. The system of Claim 41 where the distinguishing mark found for the application layer handshake is a part or all of an encryption key or enctiption certificate returned in response to a request ftr secure encrypted communication.

73. The system of any of the preceding Claims 44) to 72 wherein frequently changing parts of responses to handshake requests are suppressed or subtracted from said responses.

75. The system of claim 74 where a datastore is used to store responses or distinguishing marks resulting from more than one type of application layer handshake, or values derived fr(-)m such application layer handshakes, and associate different types of application layer handshake CSOflSCS or distinguishing marks with a common host.

76. The system of any of the foregoing Claims 40 to 75 wherein a scanning engine with access to the datastore and the network is used to initiate application layer handshakes with multiple network addresses.

77. The system of Claim 76 wherein the scanning engine is adapted to perform an automated process for associating network addresses with their hosts on a network, that comprises the fi)llOving steps: * Establishing a collection of network addresses to be scanned; * For each network address checking if a host is listening at that address; * Carrying out the minimum handshake attempt for a each given network address necessary to obtain a host response containing a distinguishing mark; * Searching the datastore to determine if it has records of a host with the distinguishing mark provided in the aforementioned host response; * If no record is found with the aforementioned distinguishing mark, creating a new host record with the given network address; * If a record is fiund with the aforementioned distinguishing mark, amending that record to reflect its relationship with the given network address.

78. An appliance connected to a computer network for identifying hosts running on the computer network in which more than one physical or virtual host is present, identifying the network address or network addresses associated with a host and establishing the relationship between identified network addresses and identified hosts, characterised iii that the appliance: * Initiates an application layer handshake with each network address to be identified; * Uses some or all of the response to said application layer handshake request to associate a network address or network addresses with a specific host.

79. The appliance of Claim 78, further characterised in that it: * identifies from the response to said application layer handshake request a distinguishing mark characteristic of the host for a network address or addresses; 4O * Compares the distinguishing mark identified for each such response to an application layer handshake with at least a second response to an applicati n lawyer hand shake; * Determines from the comparison of the responses to handshake requests whether one or more network addresses are associated with a common host.

80. The appliance of Claim 78 where the handshake comprises attempting to establish a Secure Shell connection with the network address to be identified.

88. The appliance of Claim 79 where the distinguishing mark identified for the application layer handshake is a part or all of a server certificate rcturned in response to a Transport Layer Security request.

89. The Appliance of Claim 87 where some or all of the server certificate, or a value derived from the servcr certificate, is stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the some or all of the server certificate is associated.

93. The appliance of Claim 91 where a datastore of whole or complete TELNET banner patterns, or values derived from TELNET banners, is used to distinguish between those TELNET banners returned in an attempt to connect to a network address that contain host identifying information and those TELNET banners that do not.

LL

95. Thc appliance of Claim 78 where the handshake comprises an attempt to connect to network address tO be identified using a File Transfer Protocol.

102. The appliance of Claim 10() where a datastore of distinguishing marks is used to distinguish between those root documents that contain distinguishing marks and those root documents that do not.

103. The appliance of Claim 99 where some or all of a root document returned in response to a Hypertext Transfer Protocol request, or a value derived from such a root document, is stored in a datastore in conjunction with ant' network addresses identified as corresponding to the host with which the root document is associated. 104. The apphance of Claim 100 where distinguishing marks derived from

File Transfer Protocol banners are stored in a datastore in conjunction with network addresses identified as corresponding to the host with which the distinguishing mark is associated.

106. The appliance of Claim 79 where the distinguishing mark identified for the application layer handshake is some or all of an error response to a Hypertext Transfer Protocol request.

107. The appliance of Claim 105 where a datastore of error responses, or values derived from error responses, is used to distinguish benveen those error responses that contain host identifying information and those error responses that do not.

109. The system of Claim 78 where the application layer handshake consists of a request for secure encrypted communication sufficient to cause the return of an encryption key or encryption certificate from the network address to be identified.

110. The sstem of Claim 79 where the distinguishing mark found for the application layer handshake is a part or all of an encryption ke or encryption certificate returned in response to a request for secure encrypted communication.

111. The appliance of any of the preceding Claims 78 to 110 wherein frequentk changing parts of responses to handshake requests are suppressed or subtracted from said responses.

112. The appliance of an of the preceding claims 78 to 110 in which handshakes to one or more network addresses consists of any combination of the handshakes described in Claims 80 to 110.

113. The appliance of claim 112 where a datastore is used to store responses or distinguishing marks resulting or derived from more than one tvpc of application layer handshake and associate different types of application layer handshake responses or distinguishing marks with a common host.