GB2441350A

GB2441350A - Filtering access to internet content

Info

Publication number: GB2441350A
Application number: GB0617113A
Authority: GB
Inventors: Michael John Andrew Phillips; Lee Hesselden
Original assignee: PUREPAGES GROUP Ltd
Current assignee: PUREPAGES GROUP Ltd
Priority date: 2006-08-31
Filing date: 2006-08-31
Publication date: 2008-03-05
Also published as: GB0617113D0

Abstract

Disclosed is a method of controlling access to a remote Internet site, defined by a URL. comprising the steps of: intercepting, using a local proxy server, a user's request to visit a particular site; transmitting details of the requested URL to a remote server to compare the URL with a white-list comprising details of URLs, which have previously been assessed and found to be safe; if the requested URL is present in the white-list, allowing the local proxy server to access the requested URL; or if the requested URL is not present in the white-list, directing a web-spider to visit the site and assess its content according to one or more predefined criteria, wherein if the content is assessed to be safe, the local proxy server is allowed to access the requested URL.

Description

IMPROVEMENTS IN AND RELATING TO INTERNET CONTENT FILTERING

The present invention relates to the filtering of internet content in an attempt to remove or at least reduce the risk of a user viewing or accessing offensive or inappropriate material.

There is a general concern amongst many users of the internet that they or their children may inadvertently encounter material which they would deem to be offensive or upsetting. It is not generally possible to ascertain from a web address or a URL (Uniform Resource Locator) whether the content of that particular page contains material which the user would not wish to see. There are many prior art solutions directed towards removing or reducing this risk but none of them are completely satisfactory.

In particular two common techniques for controlling which websites may be accessed are termed "black list" and "white list" In black list systems the user is prevented from accessing any websites or domain names which appear on a so-called blacklist. This type of system requires continuous updating to ensure that suspect or questionable sites are added to the black list. Given the enormous number of web pages available on the internet, the task of maintaining a black list in this fashion is not particularly practicable for most purposes. As such, the black list of sites which are deemed unsuitable is out of date almost as soon as it has been updated. This is clearly unsatisfactory as it leads to the possibility that a user could easily stumble across an offensive website which has not yet made it onto the black list.

An alternative approach uses the so-called white list.

This is also known as a "walled garden" approach and the user in these systems is only able to access websites from a predefined list of approved websites. Such a system may be suitable for a computer installation in a school, perhaps, where it may be desirable to limit the pages which are accessible to the user to a very few carefully defined suitable pages, but this is unlikely to be of any real use to most users. If a user did wish to visit a page which is not listed on the white list, then they would need to somehow request authorisation or permission from the owner of the white list so that the desired page can be verified and added to the white list. The practicalities of managing such a system on any scale would render such an enterprise too complex to be of any use.

It is an aim of embodiments of the present invention to address the shortcomings with the prior art, whether mentioned herein or not.

Throughout this description, the terms domain name, URL and website address are used interchangeably to refer to particular internet websites associated with a particular URL.

According to the present invention there is provided a system and method as set forth in the appended claims.

Preferred features of the invention will be apparent from the dependent claims, and the description which follows.

For a better understanding of the invention, and to show how embodiments of the same may be carried into effect, reference will now be made, by way of example, to the accompanying diagrammatic drawings in which: Figure 1 shows a schematic view of an embodiment of the present invention; Figure 2 shows a flow chart representing a first embodiment of the present invention; Figure 3 shows a flow chart representing a second embodiment of the present invention.

Embodiments of the present invention operate according to a dynamic white list principle. In the context of this invention, dynamic white list" refers to a database of predetermined safe websites and domain names which can be dynamically updated in real time if a user attempts to access a website not listed in the white list.

Additionally, further databases may be provided. These additional databases may include: a black list of. sites which are known to include offensive material; a list of ambiguous sites requiring human investigation; and a list of newly discovered sites awaiting automatic investigation.

Figure 1 illustrates a typical setup of an embodiment of the invention.

In order to interact with the white list, a piece of software called the client program 20 is installed on the user's computer 10. This client program includes a proxy server, which operates to intercept all requests from a s user to access the Internet 30, and ensures that all such requests are handled via the proxy server according to rules which are intended to ensure that only safe and unobjectionable sites are visited.

Once the client program 20 is installed on a particular PC, it is impossible to access the internet without such requests being intercepted by the client program, unless the user has sufficient authority to override the client program. Such authority generally belongs only to a master user who can override the security settings through use of a suitable password or other access control technique, such as a smart card or biometric identification.

In order to describe the operation of an embodiment of the invention, it is useful to examine a typical scenario.

Figure 2 illustrates this scenario in flowchart form.

If a user attempts 100 to access a website, the client program, intercepts 110 the request and queries 120 the central server 40, which compares the requested tJRL with a database, known as the white list 50, which includes IJRLs which are known to be safe.

If the requested URL is included in the white list, then a message is sent from the server 40 to the client program running on the PC 10, allowing the user to access 170 the requested site. The user will experience little or no delay in such a situation.

Additionally, if the requested URL is not included in the white list, a check may be performed to see if it features in further databases, accessible to the server. The further database may include different categories of sites, such as black-listed sites, ambiguous sites and sites awaiting further investigation, as mentioned previously.

If the requested URL features in the black-list, then access is denied and the user receives an appropriate message or is re-directed to a warning page and access is denied. If the requested URL features in one of the other databases, then a suitable message is displayed.

If the requested URL is not included in the white list 50, or any of the other lists, then this means that the requested URL is unknown to the server and the requested website is logged 130 and a web spider 60 is directed immediately to the requested URL to assess its content, as will be described shortly. The user experiences a short delay while spidering takes place 140, but this is only likely to be a few seconds.

If the assessment reveals 150 that the site is safe, then it is added to the white list 50 and the user is directed to the site, exactly as if the site was already listed in the white list.

If the assessment reveals 150 that the site is not safe for viewing, then the user is redirected 160 to an "access denied" page, informing him that the site is not viewable.

In order to assess the content of a requested site, which is not present in the white list, the web spider 60 performs an analysis of the content of the site according to one or more pre-defined rules or criteria.

There are a wide variety of criteria which may be used to assess the content of any particular website and various examples are given below: Languages If the user is an English speaker and expects to only find English language websites, then one criterion which may be used to assess a website is whether the language in which it is written is English or not. The central server may be instructed to not authorise access to any websites which are not solely or predominantly in English, simply as the analysis of a foreign language website may be too onerous.

However, multi-language analysis of websites may be available in certain embodiments of the invention.

Images / Videos Any website which consists only of images and/or video files with no accompanying text is flagged as suspicious.

This is because it is not straightforward to assess the nature of images on a website and, in the absence of any further text, it is assumed that such a website is questionable. In other embodiments of the invention, it may be possible to perform a qualitative assessment of the images using specialised software which is able to detect and analyse images for flesh tones, which may be indicative of pornographic content.

Offensive Terms Many websites which are generally considered offensive or objectionable are of the type that display pornographic words and images. Many such websites will include certain words or terms which will alert the server to the nature of the website. A predetermined list of such terms is accessible to the server and a comparison of these terms against terms appearing on the requested websites can be performed. If the requested website contains any of the defined terms or, in some cases, more than a certain number of them, then the requested website is flagged as objectionable and access is denied.

The list of offensive terms may be further classified according to a degree of offensiveness so that pages including only less offensive terms may be accessible, whilst pages including more offensive terms will be barred.

Metadata Web pages often contain a great deal of hidden information, which is not normally displayed on the user's computer when that site is visited. In particular, Metadata is concealed in the source code for web pages in an attempt to influence search engines such as Google and to provide other control information. Often, Metadata uses many of the same offensive or objectionable terms which are displayed on the user's computer and the spider 60 can form an analysis of the hidden Metadata to characterise the site as objectionable or not. Links

Another criterion which may be used to assess a requested website is whether that website contains links to other websites which are already known to be questionable, or conversely, whether there are reverse links to the requested website from websites that are known to be objectionable. In either of these cases a further S determination can be made to decide whether the requested site is itself objectionable or not. In this way, a website may first be assessed in terms of the sites it links to and are linked to it, and, if needed, a secondary assessment can be made in case of doubt.

Special Terms Under U.S. Law, pornographic websites are required to display a legal notice under USC 2257. The presence of "USC 2257", "2257" or variants thereof, generally indicate the presence of offensive material, which can be used to prevent a site being entered onto the white list. Since a great number of pornographic websites are hosted in the USA, this is a convenient method of identifying legitimate' pornography sites.

It is possible for the server to grade a website according to the degree of objectionable material contained therein.

This information may be fed back to the client program running on the user's computer and, depending on a security level set by the user, certain less objectionable websites may be viewable whereas those that are deemed to be the most offensive are not viewable. In this way, a user who is in control of the computer (e.g. a parent) may permit themselves to view websites of a kind which they would not wish or allow other users (e.g. their children) to visit.

The access level of a particular user may only be set by a user having the appropriate permissions to change various settings within the client program 20. In most cases for domestic use, the parent will be Set up to be the master user who can then assign various security levels to other users of the system to allow or disallow access to sites of different degrees of objectionability.

As can be seen from the above, a master user can override the security settings of the client program 20 so that effectively no web filtering is performed at all. The same master user may set various levels of access control so that other users of the system can enjoy varying degrees of access to websites depending upon their inclusion within the white list 50 or other lists 55,65,75,85 assessed or to be assessed by the spider.

In use, if a user not having unfettered access to all sites attempts to access a site not known to the server (i.e. not on the white list 50, or any of the other lists 55, 60, 65, 75, 85), the central server 40 dispatches the spider 60 to investigate the requested site. While this is going ahead, a warning message may be displayed to the user informing them that a site analysis is being performed and the results will be known in a short time.

Typically, assuming normal levels of network load, such an analysis will take less than a few seconds to perform. If the requested site is unobjectionable, the user will automatically be directed to the site and browse it as normal. If, however, the website is assessed to be objectionable, according to any one or more of the criteria which have been defined, then the user receives a message indicating that access to that particular website has been barred.

As a further level of assurance, it is possible to configure the system to include a degree of human intervention. If the spider is unable to determine the content of a website for certain, then it is possible to log, in an ambiguous site list 65, the URL of the website in question and schedule this for later human intervention, where a more accurate determination may be made.

For instance, depending upon the settings chosen for the server software, a website which contains nothing but images may be classified as objectionable if an extremely conservative approach is taken, or, it may be classified as suspect only, requiring human intervention to verify one way or another what the content of that website is.

If human intervention is used, the user receives a suitable warning notice informing them that the website is being checked manually and asking that they revisit that site at some point in the future to determine whether access has been allowed or not.

The extent to which human intervention is necessary or desirable depends largely upon the level of control required. If it is desired to err on the conservative side, then any site which is assessed by the spider and which appears suspect, is simply not entered onto the white list and access is thus denied. If, however, a less conservative approach is required, and, in an attempt to prevent legitimate sites from being barred, then human intervention can be used.

In a further enhancement, embodiments of the invention may be configured to make use of certain data stored locally on the user's PC, which can allow access to known safe sites without recourse to the central server 40. This may be achieved by the provision of a local database 21 including one or more lists 22, 24 of sites which are deemed safe and unobjectionable according to one or more criteria.

In particular the database 21 may consist of two lists.

The first list 22, hereafter called the quicklist, may comprise a number of websites which are assessed and determined to be safe by the provider of the system i.e. the party which provides the client program and operates the central server. Such a list may include popular websites which are known not to include any objectionable material, such as popular newspaper, entertainment, information and computing websites, to name only a few, which are generally safe and unobjectionable.

The second list 24 may include a personal cache of websites which the user has previously visited and which have been either deemed safe by the central server and/or the spider, or which the user has overridden the security settings to access. Each user registered with the client program (e.g. master user and various other users, such as children) has a separate personal cache 24 so that sites which one user has deemed acceptable may not be visited by another user without being first assessed as described.

Figure 3 shows a flowchart illustrating the operation of the second embodiment. Similarly numbered operations shown in Figure 3 are identical with those shown in Figure 2 and so are not described in detail again.

In use, when a user wishes to access a website, a check of the URL is made 200, 210, firstly against the local database 21, including the quicklist 24 and the list of sites in the personal cache 22, to determine if the website to be visited is listed amongst these. If it is, then access is allowed without recourse to the central server 50 or spider 60, since these sites have previously been deemed safe. This arrangement greatly reduces the load on the central server 50 for cases where the user wishes to access websites which have already been visited and/or judged safe.

In the event that the requested URL is not included in either the quicklist 22 or the cache of personal sites 24, then the requested URL is transmitted to the server 40 for checking as described previously.

Most users tend to regularly visit a relatively small number of websites and by use of the local database 21, in most cases, access to the central server's white list 50 will not be required, thereby reducing the load on the server and also increasing the Internet access speed enjoyed by the user.

An unauthorised user may be tempted to hack into the local database 21 in an attempt to manipulate the lists stored therein, so as to gain access to offensive sites. The local database is therefore suitably encrypted using known encryption techniques so that it is protected against such hacking attempts. The encryption is managed by the client program 20.

In addition to the dynamic white list which has been described so far, information on websites which may not be visited at all may be optionally contained in a separate black list which is kept alongside the dynamic white list.

For instance, if a particular domain name is known to include pornographic or offensive material, then this can be added to the black list, meaning that it can never be visited by a user not having the appropriate level of security clearance.

The spider program (also known as a robot or a crawler) which is used to visit and assess the content of requested websites is also configured to randomly roam the web looking for new websites which can then be assessed on a continuous basis, and used to update the white list 50. It is arranged to operate in this way at times when it is not specifically tasked with assessing a user's requested URL.

In this way, the white list may be updated continuously.

Alternatively, a plurality of spiders may be provided, each designated for a particular task i.e. one or more may be randomly trawling and another one or more may be arranged to respond to a user's request.

The granularity of the white-list can be adjusted according to particular user's needs. This means that if a sub-domain is found to contain offensive material, then either the corresponding domain can be denied entry to the white list or only the sub-domain. For instance, if the sub-domain support.xyz.com is found to contain offensive material, then only that sub-domain (support.xyz.com) can be omitted or the entire domain (www.xyz.com).

The present invention can be implemented as computer-readable code on a computer-readable recording medium.

The computer-readable recording medium is any data storage device that can store data which can be thereafter read by a computer system. Examples of the computer-readable recording medium include read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and carrier waves (such as data transmission through the Internet).

The computer-readable recording medium can also be distributed over network-coupled computer systems so that the computer-readable code is stored and executed in a distributed fashion.

In order to operate the client program on a PC, the user must install it and configure the program with suitable information identifying the master-user and any other users who are to use the PC. Different levels of access privileges may be assigned to each user, and each user will be associated with a different personal cache.

Periodically, the client program may receive software updates from the server 40. These may include updates of the operational code, the quicklist URLs or other program modules.

An ongoing subscription may be payable by the user in order to make use of the service.

Attention is directed to all papers and documents which are filed concurrently with or previous to this specification in connection with this application and which are open to public inspection with this specification, and the contents of all such papers and documents are incorporated herein by reference.

All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and/or all of the steps of any method or process so disclosed, may be combined in any combination, except combinations where at least some of such features and/or steps are mutually exclusive.

Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.

The invention is not restricted to the details of the foregoing embodiment (s). The invention extends to any novel one, or any novel combination, of the features

disclosed in this specification (including any

accompanying claims, abstract and drawings), or to any novel one, or any novel combination, of the steps of any method or process so disclosed.

Claims

CIAIMS

1. A method of controlling access to a remote Internet site, defined by a IJRL, comprising the steps of: intercepting, using a local proxy server, a user's request to visit a particular site; transmitting details of the requested URL to a remote server to compare the URL with a white-list comprising details of URLS, which have previously been assessed and found to be safe; if the requested URL is present in the white-list, allowing the local proxy server to access the requested URL; or if the requested URL is not present in the white-list, directing a web-spider to visit the site and assess its content according to one or more predefined criteria, wherein if the content is assessed to be safe, the local proxy server is allowed to access the requested URL.

2. A method as claimed in claim 1 further comprising the step of comparing the requested URL with a black list of sites to which access is barred if the requested URL is not present in the white list.

3. A method as claimed in claim 1 or 2 wherein if the assessment reveals that the content of the requested site is safe, then the URL of the requested site is added to the white list.

4. A method as claimed in claim 1 or 2 wherein if the assessment is that the content is not safe, then the user receives an access-denied message.

5. A method as claimed in claim 4 wherein the access-denied message is delivered by the user being directed to a webpage.

6. A method as claimed in any preceding claim wherein the predefined criteria by which the content of the requested site is assessed includes one or more of: the language of the content; the presence of only image and/or video files; the presence of offensive words or phrases, as defined in a database; the presence to offensive words or phrases, as defined in a database, in metadata; a link to or from the requested site to a site known to contain offensive material; and the presence of certain predefined terms, indicative of pornographic material.

7. A method as claimed in claim 6 wherein the database of offensive words or phrases is available to the server or the spider.

8. A method as claimed in claim any preceding claim wherein different users are assigned different access levels, allowing suitably authorised users to view content not available to users having a different access level.

9. A method as claimed in any preceding claim wherein, before details of the requested URL are transmitted to the remote server, the requested URL is compared with entries in a local database of IJRLs to which access is allowed, s such that details of the requested TJRL are only transmitted to the remote server if the requested URL is not present in the local database.

10. A method as claimed in claim 9 wherein the local database comprises a list of URLs which have previously been visited by the user.

11. A method as claimed in claim 10 wherein the local database comprises a further list of URLs which have been provided by a third party.

12. A method as claimed in any of claim 9 to 11 wherein the local database is encrypted.

13. A computer-readable storage medium having software resident thereon in the form of computer readable code executable by a computer for performing the method of any preceding claim.

14. A system comprising a server and a web-spider, the system being operable to: receive details of a requested URL from a user's personal computer; compare the requested URL with a database of known safe URLs; transmit a message to the personal computer, allowing access to the requested URL, if the requested URL is in the database; or direct the web-spider to assess the content of the site at the requested URL, using one or more predefined criteria, if it is not in the database.

15. A system as claimed in claim 14 further operable to compare the requested URL with a black list of sites to which access is barred if the requested tJRL is not present in the white list.

16. A system as claimed in claim 14 wherein the server is operable to transmit a message to disallow access to the requested URL if the assessment of the requested URL reveals the presence of offensive material.

17. A system as claimed in any of claims 14 to 16 further comprising a personal computer arranged to comprise a local database of known safe URLs to which access is allowed without recourse to the server or web-spider, these only being used if the requested TJRL is not in the local database.

18. A system as claimed in claim 17 wherein the local database comprises first and second lists of URLs which have been previously visited by a particular user and IJRLs which have been provided by a third party, respectively.

19. A method as hereinbefore described and having particular reference to the accompanying drawings.

20. A system as hereinbefore described and having particular reference to the accompanying drawings.