GB2412805A

GB2412805A - Detecting and recording events on a computer system

Info

Publication number: GB2412805A
Application number: GB0407344A
Authority: GB
Inventors: Jeremy Andrew Charles Barker; Carlton Lee Moore
Original assignee: OPTIC VISION Ltd; Flintrock Ltd
Current assignee: OPTIC VISION Ltd; Flintrock Ltd
Priority date: 2004-03-31
Filing date: 2004-03-31
Publication date: 2005-10-05
Anticipated expiration: 2024-03-31
Also published as: GB2412805B; GB0407344D0

Abstract

A method and apparatus for detecting and recording events on a computer system 10 is disclosed. The system 10 has a camera 62 to capture an image of the computer user, an external storage device 50 for storing the captured images and capture means 30 for capturing data entered into a computer system 12. When data entered into the computer system 12 is inappropriate, suspicious or unauthorised an image of the user 21 is acquired by the camera and stored in the storage device. In this way, users performing unauthorised actions may be identified.

Description

24 1 2805 1

DETECTING AND RECORDING EVENTS ON A COMPUTER SYSTEM

Field of Invention

This invention relates to a method or apparatus for recording computerbased events if a computer user conducts an inappropriate, suspicious or prohibited act on a computer. In particular, but not exclusively, the invention relates to recording computer data and physical data to properly identify the person inputting a data-object into a computer.

Background of Invention

Computer-based fraud or deception is an increasing problem, as is computer-based deception or abuse. For instance, abusive emails can be sent via a computer network which may damage a person or cause offence. Such emails usually include the address of the person sending the offending literature on the message sent to the recipient of the email. However, this information is not conclusive proof of the actual person who sent that email. For example, a third party may have logged onto the computer system using false identification data. As a result, emails written and sent by that third party appear to have been sent by another person. Thus, the third party can escape detection and may continue their offensive behaviour without proper detection or recourse.

Other offences may include falsification of records on a computer system, or improper use of a computer system in a public area (such as an internet cafe or public library) to - 2 - download or distribute offensive material available on the internet, for instance.

CCTV (Closed Circuit Television) surveillance systems are a well established technology which can be used to monitor activities in a room or an area for security purposes. Surveillance cameras can be used to identify persons within a room operating a computer. The image is often recorded on a video tape which is stored at a separate location from the camera. However, such surveillance systems are inadequate because they are unable to record events taking place on the computer system. The surveillance cameras only provide a visual record of persons in a room operating a computer.

A camera could be arranged to record the output from the computer monitor. However, the camera would have to be placed close to the screen to get a proper view, and it would be relatively easy to avoid the display being recorded by various methods (such as reducing the brightness of the display so that a clear record of the display is not possible, or by obscuring the display from the camera).

Furthermore, if a prohibited event has taken place, then the security services have to trawl through lengthy video tapes in an attempt to identify the perpetrator.

Summary of the Invention

The present invention aims to ameliorate the problems with known security systems discussed above. In one aspect, the present invention provides a method of recording an activity undertaken on a computer, comprising: a) comparing a data object processed by the computer with a plurality of data object classifiers stored in a pre determined object-set, said plurality of data object classifiers representing a corresponding plurality of activities; b) when it is determined by the comparing step that the data object matches a one of the data object classifiers, sending a command to initiate a camera arranged to capture a visual image of a person at the computer or the environment in which the computer is located; and c) storing at least the data object representative of one of a plurality of activities in a data storage.

The present invention further provides a computer program which, when run on a computer, carries out the method described above.

In a further aspect, the present invention also provides a method of recording a computer-based event, comprising: monitoring values input to the computer, or processed by the computer, comparing input values with a pre-determined list of value classifiers arranged to determine whether said input values indicate that a pre-determined activity is being undertaken, on detecting a match between said input values and values in said predetermined list, recording at least the input values and data from a camera arranged to capture an image of the user at the computer, or the environment in which the computer is located, and storing the recorded data in a data storage so that the camera and input values are linked with one another.

In another aspect, the present invention provides a device for recording a computer-based event and a user undertaking said event, comprising 4 - a camera arranged to capture an image of the user so that they can be visually identified, identification means for identifying when said event takes place on the computer, capture means arranged to capture data from the computer and the camera when the identification means identifies that said event is taking place so that the computer and camera data are usable to identify said event and user, and a data storage unit arranged to store data from the capture means.

The methods and apparatus have the advantage of providing a means to record data objects or events (such as alpha-numeric, a periodical internal or external trigger signal or data relating to scanned images, for instance) input into the computer, or processed by the computer, which can be used to indicate an activity which is undesirable and might cause offence, or which might be illegal or fraudulent and, at the same time, initiate a peripheral or dedicated device to visually record the person at the computer who input the information or data into the computer. The peripheral device is arranged so that it records the environment in which the computer is disposed so that it can later be established whether there was for was not, as the case may be) a person at the computer when an event took place which caused an embodiment of the present invention to record the screen and capture an image from the camera. In this manner, more conclusive proof of the perpetrator who undertakes a certain activity can be recorded; the visual recording of the person provides evidence of the actual person, and not just the person presumed to be logged-on to the computer. 5

Advantageously, the camera may be initiated when the certain event takes place. This results in a relatively short amount of video storage being required to store the visual images or video sequence. Also, the data from the S camera can be meshed or inextricably linked with at least the data object or a snap-shot of the screen to provide a combined image which shows the data object alongside visual recordings taken at the same time.

Advantageously, the aspects of the present invention provide a means for visually recording the identity of the person using the computer at the time of the prohibited event, as well as recording the prohibited event itself.

There are occasions when it might be necessary to determine whether or not a person was actually sat at the computer when an event was recorded. For instance, embodiments of the invention might be arranged so that a snap-shot of the person seated at the computer is taken on regular intervals, for instance every ten minutes during the working period. The data object in this instance is a trigger signal (either produced internally or sent from an external source to the computer) which is processed by the computer and which causes the computer to capture an image of the environment in which the computer is located. This image can be used to determine whether or not anyone was located at the computer when the trigger was fired. This might be useful if it is necessary to determine the location of a person at certain times during the day. For instance, it may be important to know whether a security guard is sat at his desk.

Other events could be used to trigger the computer, or the computer network to capture an image either of a single computer of a network or all computers on a network. For - 6 - instance, if an intruder alarm device is triggered during non-working hours, then it would be advantageous to know who was located in the building and particularly where they were located.

It is desirable to have the capability to detect certain types of events or data which are being processed by the computer during its use. For instance, if an image is input into the computer via a scanner, some form of image recognition program might be incorporated into embodiments of the present invention so that the type of image input into the computer can be classified. For instance, it might be undesirable for images of naked people to be scanned into the computer. Embodiments of the present invention could be arranged to detect such occurrences and capture an image of the person at the computer and the display screen.

Furthermore, it might be desirable to classify the types of programs being run on a computer. For instance, in a Windows(RTM)-type environment, the name or type of program being run often appears along the top bar of the application window. Embodiments of the present invention can be arranged to recognise certain terms or phrases on that top bar and, on recognition initiate a camera. Such phrases might be "gambling", or "pornography". Thus, embodiments of the present invention act as a further safety measure in tandem with any firewalls which might protect the network from undesirable activities. Firewalls can act to prevent person on a network having access to certain, predefined web pages, and to prevent certain items being downloaded on to the computer or computer network. However, undesirable web pages can be accessed if the URL has changed, or if links are provided via other web pages which the firewall software is not aware of and/or does not bar access to. As a result, 7 - a network incorporating an embodiment of the present invention can act to deter network users from accessing undesirable or unauthorized web pages, since they are aware that a record is kept of any activity taken by a person sat on a computer on which the undesirable or unauthorised activity takes place.

Embodiments of the present invention have many other potential uses where it is deemed necessary to monitor computer user activity. For instance, embodiments of the invention could be used in schools or colleges, or in the home or office environments. An embodiment of the present invention might be setup to record events which could be a completely innocent use of the computer. For example, if the embodiment of the invention looks for a sixteen digit number string (associated with a credit card ID number), then the present invention might be triggered to record events if a person is using a number-based application (such as a spread sheet). The recording should be made and reviewed at a later date to determine the exact activity of the person. To do this effectively, it may be necessary to record the values input to the computer (and not just a snap- shot of the computer display). Also, the data object from the computer (such as input values or strings) and camera data should be inextricably linked so that it is possible identify the person carrying out a certain event and the event being carried out.

Description of the Drawings

Embodiments of the present invention are now described, by way of example, and with reference to the accompanying drawings, in which: - 8 - Figure 1 is a schematic diagram showing an embodiment of the present invention; Figure 2 is a diagram showing recorded data from an embodiment of the present invention; Figure 3 is an alternative arrangement of recorded data according to another embodiment of the present invention; and Figure 4 is a schematic block diagram of a computer network embodying the present invention.

Detailed Description of the Embodiments

Referring to Figure 1, a system 10 embodying the present invention is shown in a highly schematic form. (The system is shown as a schematic block diagram of computer programs). Links between the programs and computer peripheral devices are shown. Data flows between the peripheral devices and the programs and between programs in the computer 12.

The computer 12 comprising application programs 14, 15 and 16. These programs may include a word processor spreadsheets or any type of software which can be installed or downloaded onto a memory in the computer. The computer also comprises a display driver 18 which is arranged to provide signals to driver display device 20 (such as a CRT LCD, or TFT display screen). The display allows a user to see programs which are run on the computer and information, data or strings which they input into the computer programs from peripheral input devices 22 (which may include a mouse or keyboard, or other types of input devices). The driver program 24 takes data objects or inputs from the mouse and keyboard which provides the appropriate data for the - 9 - applications to properly interpret the information input by the user 21 into the computer 12.

The computer applications can be arranged to run alongside one another. This allows the user to have more than one application open at a given time, with one application being 'active' so that data can be input by the user. In the embodiment shown in Figure 1, the application #1, 14, is active and data 26 from the mouse or keyboard can be entered by the user 21. The data 26 is interpreted by the driver 24 and appropriate information is sent to the application #1 so that the display shows the inputs from the user 21 on the display 20. This is a standard interface system between a computer and the user.

The computer also comprises a capture agent 30. This is arranged to monitor the inputs from the keyboard and to determine, from the user input, the activity which is being undertaken by the user. The capture agent determines when an event has taken place in an application (such as the user inputting a character string) and then check the event against a database or list to determine whether the event might constitute a prohibited activity. For example, if application #1 is a Word Processing application then the capture agent may look for individual words or strings of words which are considered to form part of a prohibited activity. For example, the capture agent can look for profanities or racist phrases. Also, if active application #1 were an network-based application, such as an internet search engine or an e-mail application, then the capture agent could look for a word which are associated with elicit materials, or offensive material. In the example of the application being an e-mail application, the capture agent might be arranged to look for derogatory or derisory words or phrases which could be sent to a third party and cause offence or injury.

If the application is a spreadsheet based application, then the capture agent might look for events which could be used to determine whether the user 21 is changing or amending accounting details to an extent which might be considered fraudulent.

When a prohibited event is detected by the capture agent, the capture agent starts to record computer data objects needed to provide clear evidence of any wrong doings. The recording might be done over a relatively short period of time (from a few seconds to a few minutes) to ensure a record of events as they happened is kept.

Alternatively, a snap-shot of the screen might be stored.

The recorded data object might be data objects directly from the keyboard mouse 40, data objects from the mouse driver 42, data objects from the active application 44, or data objects 46 from the display driver. These data objects should include the string which has been detected as being prohibited, and preferably any data which preceded the string. Also, data input subsequent to prohibited data object should be recorded so that a thorough record of events is kept. Data 46 from the display driver 18 would usually include the information required and other peripheral information such as the time and date. The data 46 may take the form of a screen dump of information on the display 20.

Once assimilated, this data object can be sent by the capture agent to an external data storage 50. The external storage should be arranged so that it is tamper proof from the user 21 or from anybody using the computer 12.

Alternatively, internal storage can be used to store - 11 appropriate recorded data for the purposes of identifying a prohibited act. Again, data stored here should be kept in a tamper proof environment to prevent it being erased, or amended.

At the same time that the capture agent starts to obtain the necessary information from the computer, the capture agent sends a command signal 60 to initiate a camera 62. The camera is arranged to visually identify the user operating the computer. The data from the camera can be returned to the capture agent for storage purposes, or alternatively it can be sent to the external storage 50.

Appropriate information, such as the time and date of the camera recording, should be included in the camera data so that it can be shown to correlate with the computer display captured by the capture agent.

Where the data from the camera is sent to the capture agent, the capture agent can be arranged to provide a split screen output which shows the display screen output in one window and the camera recording in the other window. Such a split screen arrangement is shown in figure 2.

A recorded screen display 70 is shown. Screen is split into two parts 72 and 74. The left hand part 72 shows the visual recording made by the camera. The camera is arranged so that a view of the user 21 is provided. This view should be of the users face to allow positive identification.

Also, in the top left hand corner of the display 72, an indication 76 of the date that the video recording was made is provided. In the right hand corner of the left hand side of the screen 72, an indication 78 of the camera number which is carrying out the recording is provided. This is useful to identify which camera recorded the event if a plurality of cameras is provided in a computer room. 12

The right hand side of the screen 74 shows an output of the computer display shown at the time that the capture agent as alerted to a prohibited event taking place. The display screen shows the applications 14, 15 and 16 which are running. Also shown in the top portion of the display screen is the identification 84 of the person logged onto the computer. In the top right hand corner of the display screen 74 is an indication 86 of the time that the alert occurred. In the lower portion of the screen 88, a section of the active application is shown and the section of text which caused the capture agent to initiate recording is highlighted. This might be useful as evidence of a prohibited event taking place, or whether the event was innocent.

As mentioned previously, this data can be recorded as a video sequence or as a snap shot. Some jurisdictions may require that a warning is provided to the user telling him that the events are being recorded and continuation would be considered in breach of local laws. In this way, the present invention provides a means to deter people from acting inappropriately and also for obtaining clear evidence which can be later used to identify a person acting inappropriately on a computer or computer network.

Referring now to figure 3, which shows various views which may be displayed on a central control unit from different cameras covering a computer room. Each of the computers might be on a network for instance and access to members is allowed to the network via computers in a dedicated suite. Four cameras cover the room and each camera shows a view of that room. The output 100, 102, 104 from 3 cameras is shown which is used to monitor the three computers within the room. The output 106 from a fourth camera can be used to provide a general view of the room which might be useful as additional evidence.

The capture agent may be arranged so that it is installed on individual computers, or on a server of a computer network. If the capture agent is installed on a server, then it should have the capability to be able to continually detect events which are taking place on each of the computers in the network. This would mean that the capture agent would most likely have to scan through each computer in turn to detect events taking place on each computer. In order to do this scanning to an appropriate extent, the number of computers on a computer network might have to be limited. For this reason, it is more preferable that capture agents are arranged on each computer terminal in a network system.

Figure 4 shows an arrangement 120 where the system is installed on a computer network 122 there a n computers in the network and each computer has a capture agent 122 installed. The computer network 124 is linked to a server 126. A recording unit 130 is also linked to the network 122. This recording unit is further linked to an array of cameras 132 and an array of alarm devices 134. When the capture agent identifies that a prohibited event is taking place, a command signal is sent to the recorder unit 130.

The recorder has a dedicated display 136 which shows instant captured data from the computer on which the capture agent has been activated. The recorder 130 also initiates recording of video data from an appropriate camera in the camera array 132 so that physical identification of the person operating the computer on which the capture agent has been initiated is recorded.

Embodiments of the present invention can be arranged to look for and identify data objects comprising number strings, as well as word strings. For instances, it might be desirable to identify a moment when a creditcard transaction takes place. In which case, the capture agent should be able to look for a sixteen digit number string denoting the identification code on a credit-card. Of course, there might be some misinterpretation if the user is (innocently) using a number-based application, such as an accounting package or calculator. In this instance, the event should be recorded for later review, but it might not be necessary to give an on-screen warning to the user because of the potential for innocent use.

The capture agent might also incorporate software which is able to determine what the user of the computer is inputting into the program being run and/or also what other users are inputting in response to the current user's activities. For instance, if the current user is operating in an internet chat-room type activity on the computer, then the incorporated software could be arranged to monitor what is input by people in response to the user's messages. This capability is seen as having applications in protecting people from so-called "grooming" by a third party using an internet chat-room (for instance).

Such grooming activity usually takes place over a long period of time and after several conversations in the internet chat-rooms. Thus, the software should be able to look for certain word patterns being used by either party undertaking conversations with one another in a chat-room.

Such word patterns might include "let's meet", "secret", or any other phrase which may be associated with a grooming activity. -

Of course, these patterns of words may not be used at the same instance or during a single session on the chat room (or completely innocently). As a result, the computer program should look for a pattern in the use of certain words or phrases over a period of different chat-room sessions, and when the word-use exceeds a predetermined threshold, an alarm should be raised. The alarm might take form of an e-mail to a person (possibly parent or guardian) who is responsible for the user of the computer.

Various levels of thresholds can be set for different categories of significance of the event (or suspicious event) which is recorded. For instance, a low level category might be given to the use of certain data being processed by the computer which is most likely to be innocent. Various categories of increasing severity can be set by the network manager or person responsible for the computer. As a result, a high degree of control is provided.

The capture agent can be triggered by any data object input into the computer or processed by the computer. For instance, the data object might be an internal timer set to trigger the capture agent on a regular basis. At each trigger, the capture agent can capture an image from a camera and, if desirable, an image of the screen display of a computer. Other data objects might be triggers from movement sensors which are activated when a human enters or leaves an area.

The applications in which embodiments of the present invention might be useful are numerous. For instance, embodiments of the present invention might be useful in a warehouse environment where stock is logged in and out of the warehouse by a computerized system. The capture agent - 16 can be arranged to capture an image on the camera and of the display screen every time an item is logged out of the warehouse. Such an arrangement might result in a large number of triggers and resulting data to be stored for later use, if necessary. As a result, it might be advantageous to set various threshold levels, depending on the value of the item or items being logged out of the warehouse. Various degrees of importance can thus be tagged to each data file containing the captured screen display and image of the person, or environment in which the computer is located.

This arrangement should provide a clear indication of who has been responsible for logging various items into or out of a warehouse, thus providing a clear audit trail for every item in and out of the warehouse via the computer system and a record of who was responsible for logging the items in or out.

Various other software modules can be incorporated with the embodiments of the present invention. These may include face recognition software which can be used in conjunction with the camera and the captured image of the user to identify (or provide a degree of recognition) a person using a computer system. Thus, a system might be envisaged which incorporates an embodiment of the invention and such face recognition software to prevent unauthorized persons from carrying out certain activities. For instance, the system could be used to prevent an unauthorized person from logging items out of the warehouse in the example discussed above.

In a further embodiment, an additional layer of security can be encapsulated in to the system. When the computer system requires a unique log-on identification code for authorised users, a picture of the authorized user can be displayed on the system administrator's screen alongside - 17 an image from the camera of the person at the computer. In this way, the administrator is able to at least recognise whether the person logged-on to the network system is indeed the person sat at the terminal. Alternatively, or additionally, facial recognition software could also be incorporated to raise an alarm if the face of the person sat at a computer does not match the face associated with the authorised user on a data-base of authorised users (to which the facial recognition software has access).

Of course, there are many other applications in which embodiments of the present invention can work to help prevent or deter unauthorized or undesirable computer activities in a commercial or domestic environment.

Further embodiments of the present invention are obvious to the skilled person. For instance, an embodiment of the invention may be installed on a home computer for parental control. The camera might be a web-cam arranged for normal interned video use. However, if an inappropriate action is taken by a family member then the web-cam can record the person sat at the computer and the capture agent can record the events which are prohibited. Of course, various levels of prohibition can be set by either the server of computer manager so that certain levels of control are available. Also, the capture agent might take the form of software or hardware. - 18

Claims

CLAIMS: 1. A method of recording an activity undertaken on a computer,

comprising: a) comparing a data object processed by the computer with a plurality of data object classifiers stored in a pre determined objectset, said plurality of data object classifiers representing a corresponding plurality of activities; b) when it is determined by the comparing step that the data object matches a one of the data object classifiers, sending a command to initiate a camera arranged to capture a visual image of a person at the computer or the environment in which the computer is located; and c) storing at least the data object representative of one of a plurality of activities in a data storage.
2. The method according to claim 1, wherein data from the camera is stored with the data object.
3. The method according to claim 2, wherein the data from the camera and the data object are linked to one another.
4. The method according to claim 1, further comprising storing data from the computer for identifying the time and date of the undertaken activity.
5. The method according to claim 2 or 3, wherein the stored camera data is arranged so that, when replayed, it appears with the stored data object. 19
6. The method according to any preceding claim, further comprising displaying a warning on the computer's display if the data object correlates with at least one of the plurality of data object classifiers.
7. The method according to any preceding claim, wherein the computer is disposed on a network of computers, the data object is processed by a first computer on the network, and the comparing step is carried out by a second computer on the network.
8. A computer program which, when run on a computer, carries out the method according to any preceding claim.
9. The computer program according to claim 8, wherein the program is disposed on a first computer in a network of computers.
10. The computer program according to claim 9, wherein the data storage of claim 1 is disposed on a second computer in the network.
11. A method of recording a computer-based event, comprising: monitoring a data object input to the computer, or processed by the computer, comparing said data object with a pre-determined list of data object classifiers arranged to determine whether a pre-determined activity is being undertaken, on detecting a match between said input data object and data object classifiers, recording at least the input data object and data from a camera arranged to capture an image of the user at the computer, or the environment in which the computer is located, and storing the recorded data in a data storage so that the camera and input values are linked with one another.
12. The method according to claim 11, wherein the recorded data includes computer display data.
13. The method according to claim 11, further comprising storing data from the computer for identifying the time and date of the activity being recorded.
14. The method according to claim 11, further comprising displaying a warning on the computer's display on detecting a match.
15. The method according to claim 11, wherein the computer is disposed on a network of computers, the input data object being processed by a first computer on the network, and the comparing step is carried out by a second computer on the network.
16. The method according to claim 15, wherein the data storage is disposed on either the first or second computer in the network.
17. The method according to claim 7 or 15, wherein the camera is disposed on the network. - 21
18. A device for recording a computer-based event, a user undertaking said event, and/or the environment in which the computer is disposed, comprising a camera arranged to capture an image of the user so that they can be recognised, or the environment, identification means for identifying when a one of a plurality of events takes place on the computer, recording means arranged to record data from the computer and the camera when the identification means identifies that said event is taking place so that the computer and camera data are usable to link said event and user or environment, and a data storage unit arranged to store data from the recording means.
19. The device according to claim 18, wherein the computer is disposed on a network of computers.
20. The device according to claim 19, wherein the storage unit is disposed on a second computer in the network.
21. The device according to claim 18, wherein the identification means is disposed on the computer on which said event takes place.
22. The device according to claim 21, wherein the recording means is disposed on a second computer in the network or on the network's server.

19259B:CRC:SDS