KR20020035840A - Method of surveilling internet communication - Google Patents

Abstract

A network probe terminal for tracking a network communication line and simulating a browser activity of a given terminal. The probe terminal monitors TCP/IP data packets routed through the communication line for filtering relevant requests and responses rotating to a given IP address. These requests and responses are analyzed and sorted according to their type and content. Based on the analysis, the probe terminal identifies all relevant data transactions relating to the navigation process of the given terminal. The probe terminal activates a virtual browser simulating the processing of identified data transactions to create navigation presentations similar to the real navigation as seen by the user of the given terminal.

Description

How to Monitor Internet Communications {METHOD OF SURVEILLING INTERNET COMMUNICATION}

Today, huge amounts of traffic pass through computer networks, some of which are harmful. Therefore, the owner or manager of a particular network may be interested in tracing or “listen in” in real time to effectively monitor and / or protect the network. Such monitoring or monitoring may be accomplished by connecting a probe to the network to monitor data between two of many nodes (eg, user workstations) on the network.

In a system where communication between two nodes is in the form of discontinuous packets, the network probe may "read" the packet of data or the protocol of the packet to gather information such as about the source or destination address of the packet. It is also possible to calculate statistically relevant information, such as the average or total amount of traffic of any protocol type over a period of time, or may transmit the total number of packets to or from the node. This information may be reported to the system administrator in real time or stored for later analysis.

Various attempts have been made to this trend. For example, Clear View Network Window, a software program available from Clear Communications Corporation (Linclonshire, Ill., USA), provides so-called predictive and proactive management, intelligent root cause analysis, and quality assurance reports. However, its output is designed for network fault management and is not the same as "tapping" for communication between nodes in a network. Thus, the Clearview system does not allow monitoring of data transmitted between two nodes on the network for content or characteristics.

The Livermore National Laboratory (Livermorem, Cal., USA) "sniffes" data packets passing through a local area network, protecting computers in the US Department of Energy. A program group was developed. The US Navy used one of these programs, known as the "iWatch" program, to intercept communications from suspected computer hackers who had invaded computers at the US Department of Defense and NASA. The iWatch program uses a network probe to read all packets through the network, and then stores this information in a common database. A simple computer program can then be written to read out the stored data and display only a predetermined "interest" portion of the information.

Whenever a portion of the information of interest is found, the stored data is rescanned to inform the particular number of characters located on either side of the "interesting" portion. These characters of interest are then reviewed to determine the content of the message and used as a guide to future monitoring actions.

This system is limited to the historical analysis of user activity and does not perform a complete "tapping" of all user activity and a complete simulation of user surfing activity.

There are three major problems with how to achieve continuous and reliable tracing:

(a) Individual browsers do not report all activities performed to the web server. For example, when a browser loads a web page from its browser cache space or from a proxy server, the browser does not send a request to any "remote" web server via the cyberspace autostrade.

(b) Because browser interface mechanisms are different and proprietary to each other, applications designed to perform certain features by one manufacturer's web browser are generally not compatible with ones produced by another vendor.

(c) Individual browsers send their requests to the web server in an unorganized way. In other words, for a given web server, the preceding request is independent of subsequent requests. In request processing, the web site does not control the order of the requests.

To address these issues, US Pat. No. 5,951,643 discloses a mechanism for reliably organizing and managing information about web synchronization and tracking between multiple consumer browsers.

However, this solution is limited to the tracking activity of identified users who agree and are willing to cooperate with "tapping" and connected to designated applications and hosts.

Accordingly, it is a primary object of the present invention to provide a monitoring and supervision method and system for a network communication provider to tap connected users into a network.

Another object of the present invention is to provide a tapping methodology that allows a network communication provider to monitor in real time all user activities while communicating with the network.

Another object of the present invention is to allow web site owners to monitor and tap users who access their web sites.

The present invention relates to a method and system for monitoring and monitoring network communications by analyzing data over network communications.

These features and advantages of the present invention will be more clearly understood from the following description of some preferred embodiments given by way of example with reference to the accompanying drawings.

Figure 1 shows a typical network configuration to meet the present invention.

2 shows a terminal agent overview of the operation.

3 shows a process for tracing and identifying TCP / IP data packets.

4 is a flowchart for classifying TCP / IP requests.

5 is a flowchart that simulates the generation of a virtual secondary TCP / IP request.

6 shows a process for simulating original browser activity.

According to the present invention, a method of trekking a network communication line by a network probe terminal (terminal agent) simulating browser (original browser) activity of a predetermined terminal, the method comprising: accessing the network communication line, the communication line Tracing a TCP / IP data packet routed through a network, selecting a TCP / IP data packet (identified data packet) for a given IP address, and presenting requests for new access from the identified data packet ( Original requests), a current web page component (new navigation component) representing new addresses from the identified data packet, the new navigation component in two categories-the categories are embedded objects Or frames (false new components) Dividing the original requests into primary or secondary requests based on a criterion, wherein the original requests are true to the new component. Or the original requests do not match any new connection component and belong to the HTTP or POST type; The original requests match the false component or the original requests do not match any new connection component and do not belong to the HTTP or POST type-an HTML data file relating to the primary requests from the identified data packet , Each of which is a primary response, generating virtual secondary requests in accordance with each primary response, and responding to the virtual secondary requests from the identified data packet. Each being a secondary response), and simulating a web page presentation on the terminal agent according to the respective secondary responses.

1, the terminal 01, 02,... Is connected to the same communication line, and the communication line is used as an internal network (intranet) or an external network such as the Internet. According to the present invention, it is proposed to connect a designated network probe (hereinafter referred to as "terminal agent") to the data communication line. Alternatively, terminal 01, 02 and the like may be connected to other data communication lines or deployed in another local network.

A general overview of the terminal agent operation is described in FIG.

The terminal agent is exposed to every data frame passing through the communication line. The data frame may include information transmitted between terminals or between external data transmissions to an external source such as an internet server.

There is an interest in monitoring the actual communication activity of a given terminal when the "owner" of a data communication line, such as an ISP or private network, surfs the Internet in real time.

The operation of the terminal agent first analyzes the data frame tracing the TCP / IP data packet. As shown in Fig. 3, data analysis is handled according to another protocol classification scheme (see RFC 0793 of the Internet Protocol), i.e. first analyzing the local network protocol and filtering external data transmissions (gateway level), IP) and finally detects a "host level" Transition Control Protocol (TCP) data packet.

By analyzing the IP header of the data packet, the IP address of the requesting terminal and the message destination is identified. The owner of the communication line can easily associate an IP address with the user terminal. Therefore, the next processing may suggest to filter out all other irrelevant data packets and to transmit data of one selected terminal (hereinafter referred to as "identified data packet").

The identified data packets are further analyzed in accordance with the RFC 079 standard, which allows full management and control of the data communication port.

According to known routines for managing TCP data communication ports, the terminal running the browser is the source of all data communication, as handled by a conventional browser, for example Internet Explorer. For example, YAHOO! The terminal with the request for the home page connects to the YAHOO! Suppose you send a request to a server. In response, the server sends an HTML data file that contains all the information of the yahoo home web page component. Thus, the browser sends a new request to receive all the components of the web page by opening a new communication "virtual" port, each port being used to send different components of the same web page. All data requests and "outsider" terminals exposed in each response are, for example, perfect YAHOO! It is not possible to distinguish between an initial "primary" request that requires a home page and a "secondary" request to receive its components. When simulating the activity of the original browser by the "outsider" survey terminal, it is necessary to identify the primary request, for example.

4 shows a process for distinguishing a primary false positive from a secondary request. Initial requests originate from different actions, such as entering a new URL by the user, selecting a hyperlink or the like. Thus, to detect the same one must analyze previous information sent to the same IP address. All new navigation components (addressing browsers to new locations) of web pages received by the terminal are classified according to their type, and all embedded objects, frames, etc. are displayed as "false" components, while hyper The link is marked as a "true" component. All data is stored in the incoming response buffer database for later use.

When identifying a request for a new connection according to the TCP analysis, the request is checked according to each navigation component (RNC) in the incoming response buffer. If the RNC is marked as "false", the request is ignored; If there is no RNC associated with the request, the connection type must be identified. If the connection is of HTML type or "post" type, it is classified as a primary request.

In order to see and observe the movement of the terminal, all "first" browser activity must be reconfigured. For this purpose, it has been proposed to use "virtual" browsers. This virtual browser has all the capabilities of a "real" browser to download web pages in real time from the Internet. However, the connection over the Internet is virtual in that there is no actual data exchange with the Internet server, but only simulating the activity of the original "real" browser.

The first function of the virtual browser is shown in FIG. The browser receives all primary requests of the "real" browser. These primary requests and each initial response from the Internet are analyzed and processed according to normal browser operation. However, the result of the secondary virtual request (in a conventional browser used to complete the process of downloading web page components) is not typically sent directly to the appropriate server via the Internet, but the virtual "secondary" request buffer data. Stored in the base.

Even if the virtual browser connection is not "real", all TCP protocol management to open and control the port connection is handled by the terminal broker as if the connection is "real".

The final process of simulating and serving web pages in a virtual browser is further illustrated in FIG. All original secondary responses coming through the communication line are analyzed and recorded in the incoming response buffer database. The virtual request is compared to each secondary response stored in the incoming response buffer database in arrival sequence. If each secondary response already exists in the buffer, these responses are sent to the virtual browser and perform processing (according to normal browser operation) to provide a virtual image of each web page component. As a result, terminal intermediation simulates in real time the exact process of downloading an Internet web page as performed by the original terminal.

If each response does not appear in the incoming response buffer database, the movement of the original local cache is estimated. If the original local cache was not used in terms of the virtual request, it is held in the buffer database until each response of the original secondary is reached. In contrast, if the actual local cache was used in connection with the response, then the local cache of the virtual browser is examined, and if each secondary response is present in the local cache, each response is sent to the virtual browser and processed as described above. . If each response does not exist in the virtual cache, one of the following alternatives may apply. According to the "passive" version of the terminal broker, no further action is taken to find a "missing" response, and the "error" message will appear on the agent terminal instead of the web page component present on the actual terminal. . According to this version, the simulation of the actual terminal is not perfect but the tapping of the movement cannot be detected. According to another "active" version, the terminal broker requests a "missing" response by addressing the web page server. Although this version may allow the terminal broker to provide a more accurate picture of the actual terminal activity, more experienced terminal users who can detect the tapping activity are traceable.

According to a further mode of implementation of the invention, tracing and tapping of relevant web page data packets as well as related message data packets, for example e-mails or chats, is provided. In order to perform such tapping, the methods and principles as described above apply to requests for sending and receiving messages over a network other than a request for a web page. The process of analyzing such requests and their responses becomes more reasonable because there is no need to check cache memory activity and is always updated by the definition of drone information.

Finally, it should be appreciated that the embodiment described above relates to an internet communication environment. However, the present invention is equally applicable to general computerized network communications such as topologies, cellular, etc. in a broad sense.

Although the foregoing description contains many details, these should not be construed as limitations on the scope of the invention, but rather as examples of preferred embodiments. Those skilled in the art will consider other possible variations within the scope. Accordingly, the scope of the invention should not be determined solely by the described embodiments, but should be determined by the appended claims and their legal equivalents.

Claims (18)

A method of trekking a network communication line by a network probe terminal (terminal agent) simulating browser (original browser) activity of a given terminal, the method comprising: Accessing the network communication line, Tracing a TCP / IP data packet routed over the communication line, Selecting a TCP / IP data packet (identified data packet) for a predetermined IP address, Selecting current requests (original requests) for a new connection from the identified data packet, Selecting a current web page component (new navigation component) representing new addresses from the identified data packet, Dividing the new navigation component into two categories, the categories comprising embedded objects or frames (false new components) and hyperlinks (true new components), Dividing the original requests into primary or secondary requests according to a criterion, wherein the criterion is that the original requests are indeed matched to the new component or the original requests are not matched to any new connected component. Belongs to the HTTP or POST type; The original requests match the false component, or the original requests do not match any new connection component and do not belong to HTTP or POST type; Selecting HTML data files (each of which are primary responses) relating to the primary requests from the identified data packet; Generating virtual secondary requests in accordance with each primary response; Selecting responses from the identified data packet regarding each of the secondary virtual requests, each being a secondary response, and Simulating a web page presentation on the terminal agent according to the respective secondary responses Trekking method comprising a. The method of claim 1, Selecting virtual secondary requests (non-responsive secondary requests) that do not currently match each of any original responses, Retrieving a local cache (virtual cache) in the form of the content of the terminal agent when an original browser local cache (original cache) is used for the non-responsive secondary requests, Simulating a web page presentation on the agent terminal in accordance with the data if the virtual cache includes data regarding the non-responsive secondary requests, and Displaying error messages at each location on the agent terminal if the virtual cache does not contain data relating to the unresponsive secondary requests. Trekking method comprising more. 3. The method of claim 2, wherein if the virtual cache does not contain the data, it is addressed via the communication line to a suitable internet server to receive and indicate respective simulated responses regarding the non-responsive secondary requests. The trekking method further comprises a step. The method of claim 1, Selecting data (eg, e-mail; message data) about a network message from the identified data packet; Converting the message data into a text data file, and Displaying the text data file on the terminal agent Trekking method comprising more. The method of claim 1, wherein the network is a local area network (LAN), and the terminal is connected to an extended communication line. The method of claim 1 wherein said communication line is an external communication line, such as a telephone line, an IZDN line, an optical line, or the like. 2. The method of claim 1 wherein said network is a LAN and said terminal agent is located at a location different from said predetermined terminal location. The method of claim 1 wherein said predetermined IP address is identified by a communication line provider (ISP). The method of claim 1 wherein the predetermined IP address is identified by a communication line owner. The method of claim 1 wherein the predetermined IP address is the address of a website visitor and is identified by a website owner. A network probe terminal for tracking a network communication line and simulating browser (original browser) activity of a given terminal, Connecting means for accessing said network communication line, Monitoring means for tracing TCP / IP data packets routed through the communication line, A first for selecting web connection components (new navigation components) and new connection requests (original requests) indicating new addresses from TCP / IP data packets (identified data packets) for a given IP address; Filtering module, First sorting means for dividing the new navigation components into two categories, the categories comprising embedded objects or frames (false new components) and hyperlinks (true new components) , Second sorting means for dividing the original requests into primary or secondary requests based on a criterion, wherein the criteria are true that the original requests match the new component or that the original requests are any new connection component. Does not match and belongs to the HTTP or POST type; The original requests match the false component, or the original requests do not match any new connection component and do not belong to the HTTP or POST type- A classification module for selecting an HTML data file (primary responses) for primary requests from the identified data packet; A request generation module for generating virtual secondary requests in accordance with each primary response, A secondary filtering module for selecting responses (secondary responses) for secondary virtual requests from the identified data packets, and Display means for simulating a web page presentation on the terminal agent according to the secondary responses Network probe terminal comprising a. 12. The network probe terminal of claim 11, wherein the filtering module comprises means for selecting virtual secondary requests (non-responsive secondary requests) that do not currently match each of any original requests. 12. The network probe terminal of claim 11, further comprising a cache module for activating a terminal agent local cache when the original browser local cache (original cache) is used in response to the non-responsive secondary requests. 14. The search module of claim 13, further comprising: a search module for addressing an appropriate internet server via the communication line and for receiving each of the simulated responses regarding the unresponsive secondary requests if the virtual local cache does not contain data. Network probe terminal further comprising. 12. The method of claim 11, wherein the data relating to network messages (e.g. e-mail; message data) is selected from the identified data packet, the message data is converted into a text data file, and the text data file is displayed on the terminal. The network probe terminal further comprises an electronic message module for. The network probe terminal of claim 11, wherein the predetermined IP address is identified by a communication line provider (ISP). 12. The network probe terminal of claim 11, wherein the predetermined IP address is identified by a communication line owner. The network probe terminal of claim 11, wherein the predetermined IP addresses are addresses of web site visitors and are identified by a web site owner.