GB2460045A

GB2460045A - Analysing multiple data sources for a user request using business and geographical data, with selected rule sets to filter the data on the databases.

Info

Publication number: GB2460045A
Application number: GB0808657A
Authority: GB
Inventors: Julian David Oates; Ian Matthew Haynes
Original assignee: Triad Group PLC
Current assignee: Triad Group PLC
Priority date: 2008-05-13
Filing date: 2008-05-13
Publication date: 2009-11-18
Also published as: GB0808657D0

Abstract

Disclosed is a method of analysing data from multiple sources. The method has the steps of accessing the data sources, receiving a first user input defining a business type, receiving a second user input defining a geographic area, and filtering the data sources to create a subset of the data sources. Next a rule set is selected, according to the first user input and applied to each data source of the subset of data sources to extract relevant information. The results are then presented as an output including some of the data sources of the subset of data sources and the extracted information. The rule set may be adapted based on the user inputs. A text query may be entered as an additional user input to search the subset of sources. Also disclosed is a computer readable medium with a program for analysing multiple sources and system for executing the program.

Description

DESCRIPTION

ANALYSING DATA SOURCES

This invention relates to a method of, and system for, analysing data sources.

The expansion of technologies such as the Internet has provided individuals and businesses with free and ready access to a large amount of information. As quickly became apparent during the early growth of the Internet, a significant problem would be the finding and accessing of the desired information. To this end, search engines of various types have been developed, which allow users to frame queries and provide results to users, based upon the query chosen. A very large body of work has gone into the optimisation of search engines. Currently the market leader in the text search of websites is 000gle. This company, like its competitors, has access to a very large processing capability and uses a series of algorithms to categorise data sources such as websites and to match queries to the data sources.

Considerable work is being undertaken to provide the most relevant results to a user based upon the text query entered. Other types of searching are known, such as the matching of text queries to pictures.

In general, all of the known existing search interfaces provide results that, in a significant number of cases, are not relevant to the query provided by the user, and/or provide the results in such a manner that they do not include sufficient information about the results returned.

It is therefore an object of the invention to improve upon the known art.

According to a first aspect of the present invention, there is provided a method of analysing data sources comprising accessing multiple data sources, receiving a first user input defining a business type, receiving a second user input defining a geographic area, filtering the multiple data sources by the user inputs to create a subset of data sources, selecting a ruleset according to the first user input defining the business type, applying the selected ruleset to each data source of the subset of data sources to extract information relating to the data source, and presenting an output comprising at least some of the data sources of the subset of data sources and respective extracted information.

According to a second aspect of the present invention, there is provided a system for analysing data sources comprising a processor arranged to access multiple data sources, receive a first user input defining a business type, receive a second user input defining a geographic area, filter the multiple data sources by the user inputs to create a subset of data sources, select a ruleset according to the first user input defining the business type, apply the selected ruleset to each data source of the subset of data sources to extract information relating to the data source, and present an output comprising at least some of the data sources of the subset of data sources and respective extracted information.

According to a third aspect of the present invention, there is provided a computer program product on a computer readable medium for analysing data sources, the product comprising instructions for accessing multiple data sources, receiving a first user input defining a business type, receiving a second user input defining a geographic area, filtering the multiple data sources by the user inputs to create a subset of data sources, selecting a ruleset according to the first user input defining the business type, applying the selected ruleset to each data source of the subset of data sources to extract information relating to the data source, and presenting an output comprising at least some of the data sources of the subset of data sources and respective extracted information.

Owing to the invention, it is possible to provide a method of analysing data sources that will provide a relevant series of results, with information about each returned data source that is relevant to the user. The user specifies a business type and geographic locality and all of the data sources founding matching these inputs are processed according to a ruleset that is designed for the specified business type. This ensures an appropriate and significant result is returned to the user. In addition to business type and geographic location, the method can further comprise receiving one or more further user inputs defining one or more further features for filtering the multiple data sources. This allows the user to hone down their selection by other factors, for example relating to the size of the business or other qualities such as awards won or the like.

Advantageously, the method further comprises adapting the selected ruleset according to a received user input, prior to applying the selected ruleset to each data source of the subset of data sources. While each io business type will have a predefined ruleset, either at a general level or at a very specific level, the advanced user can nevertheless adapt the ruleset to be used, to increase the efficiency of the process and to allow the user to make the search more relevant to their needs. For example, a user may wish to search restaurants. Part of the standard ruleset for restaurants may include a query about organic food. However, the user may have no interest in this specific area, and can remove the rule from the ruleset that is to be used.

Similarly rules can be added by the user to the ruleset.

Preferably, the method further comprises applying a standard ruleset to each data source of the subset of data sources to extract information relating to the data source. In addition to the specific ruleset for the business type, additional useful information can be retrieved from the data source by using a standard ruleset, which may extract information, for example, about the complexity of the data source (in the case of a website, for example, how many pages the website currently has). The output of the application of the standard ruleset to the subset of data sources is included in the final output to the user.

Ideally, the method further comprises receiving a user input defining a text query, searching the subset of data sources for the text query, and presenting an output comprising the results of the search. In addition to providing the user with a search result, the interface to the result can allow the user to perform free text searching on only the specific data sources that have been returned in the final output. This greatly enhances the user's ability to find the information or data source that they are actually looking for.

Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which:-Figure 1 is a schematic diagram of a system for analysing data sources, Figure 2 is a further schematic diagram of the system of Figure 1 in a wider system, Figure 3 is a flowchart of a method of operating the system, and Figure 4 is a screenshot of an output to a user.

The system of the present invention is addressed to the business problem of finding information on the Internet, which usually requires a user to submit searches to one of the established search engines. The search engine will find matches in its index and present the results to the user in some form of ranked order. The search options presented are usually!!your country!! or everywhere. If the user is trying to compare the service or product offerings between organisations, there may be specialist comparison sites for that marketplace. However, if the user wishes to undertake a comparison between sites not served by specialist comparison sites, there is no established solution.

The solution provided by the present invention, in one embodiment, is a geospatial application able to hold the details of businesses and display them on a map. The details include analytical data deduced from the content of the website. This is in addition to more traditional data such as the number of employees, turnover and business sector data. Users of the system are able to select businesses, through geography and business sector and then request comparison data based upon analytics from the selected websites. In addition, users can perform free text searches against the selected websites.

The architecture of the solution, shown in Figure 1, comprises various major components. A data capture module 2 is utilised. This module 2 visits the website of the targeted businesses 1 and extracts a copy of each webpage. These pages are temporarily stored in a datastore 3 and forwarded for processing. There may be multiple modules 2 in operation at any time.

An analytics module 4 reads rules and configuration data from a datastore 7. Using these rules, the module reviews each page and extracts and processes information that it has been programmed to identify, creating a summary for the website. This information is stored in a datastore 6. There may be multiple modules 4 in operation at any time. An analytics programming module 5 allows the user to specify the type of information to be targeted in the analytics module and how to process this data. These rules and io configuration data are held in a datastore 7.

An indexing engine 9 creates an index of the textual content for the website and stores it in a datastore 8. The indexing engine such as that contained in SQL Server provides both exact matches and "stemmed" matches i.e. words with the same linguistic stem, for example fishes, fishing, have the same stem "fish".

A search processing function 10 is invoked by a user request 11. The user request contains the text to be "matched" i.e. exact match or "stemmed" match and the group of websites to be searched. The search processing function submits the search to the index in datastore 8. The datastore 8 returns the results the web page where the text can be found. The search processing function returns the results for those websites contained in the user request.

The system of Figure 1 provides the ability to compare a user selected group businesses in similar business sectors using analytical information, including "non-price" information derived from their websites. Additionally, the ability to perform free text searches across a "specified group" of websites is provided by the system.

Websites contain large amounts of data explicit, implicit and derivable.

This data might include explicit Information about products, services, prices, office addresses, contact details, management personnel, financial reports, case studies and customers. The implicit information may relate to structural Information, such as the number of pages in a website, the types of pages, and the technology used in the site. The derivable information might be information such as maximum/minimum/average price, number of items, number of occurrences of specific items and presence or absence of a specific object.

Analysis of this data enables a profile of an organisation, as embodied in the data source (website) to be compiled. Comparing profiles of organisations in similar organisations may facilitate decision making or prioritising of one action over another. In order to create a company profile for use in the system various data needs to be harnessed and categorised. A company profile, in a preferred embodiment comprises two components, a core analytics common to all companies and a sector specific analytics, common to companies in that sector.

The core analytics comprise those analytics common to all companies.

They might indicate the degree of sophistication of the business (i.e. the larger and more complex the website, the more sophisticated the business). Core analytics can comprise the number of pages in site; maximum depth of site; presence of key pages: about us; contact us; site map; history; management; total size of site, and the technology used (plain html etc.). Whilst the detail of core analytics may be of interest to some users, in general it is easier to provide a numerical "score" derived from the core. One example might be to use the formula website score = [Number of pages in site x Maximum depth of site] + [2 x number of key pages].

The sector specific analytics represent those analytics that are common across a particular business sector. For instance, in the restaurant sector users might be interested in the following Awards (Michelin stars etc), number of covers, is the menu on the website?, average price of a starter, average price of main course, average price of dessert, is there a wine list on the site?, lowest priced bottle of wine, most expensive bottle of wine, number of different bottles of wine and the types of wine available.

When defining and/or configuring analytics, the analytics for each business segment will be different, though have some similarities. The analytics for the hotel sector will focus upon rooms, room prices and services but in structure be analogous to restaurants.

Analytics are defined in the analytics programming module 5. The analytics programming module 5 comprises a list of configurations, typically by business type; a list of rules / processing algorithms; and a list of rules used in a specific configuration. This module 5 provides the ability to create an analytic by defining/selecting the following: the configuration identification; the object to be found (using pattern matching against a string of characters or numbers) or semantic process; and the processing to be conducted on the object or group of objects. For instance this may be find the numeric value of the item; find the minimum; find the maximum; find the average; find the sum; find the number on the list; find the standard deviation of the entries; find the type of item (based on an business specific list of types); and/or find the presence of an item.

is Figure 2 illustrates the embodiment of the system of Figure 1 in a wider context. The data sources 1 that are to be analysed are connected to the Internet 14, as is a client device 12. The database 7 contains rulesets 13 that are used in the processing of the websites 1. The user of the client device 12 will frame a query that includes both a business type and a geographical location as the basis for the query. The service offered by the system of Figure 1 is either provided on a user request basis of processing the query received, or the query results in the recall of pre-processing that has been carried out on the data sources 1.

The method of analysing data sources (such as the websites 1) is summarised in Figure 3. All of the steps of the method may be carried out at the time that a user submits a query, but in the preferred embodiment, as described above with reference to Figure 1, the examination of the data sources is advantageously carried out in advance, and only the generation of the final output is actually carried out in response to the user query.

The process comprises, in step Si, the accessing of multiple data sources. These data sources could be websites or could be other specific data repositories such as databases or files etc. The accessing could be carried in an automated fashion by some form of web-crawling technology that will access websites automatically, and either copy pages for later processing, or will extract information during the process of access the website. The next step is the step S2, of receiving a first user input defining a business type, and receiving a second user input defining a geographic area. The user who is making the query will specify the type of business in which they are interested, such as restaurants, and will also define an area. This definition of an area could be done by stating a postal or zip code, or inputting the name of place or wider area such as a county.

io Once the user inputs have been received, then at step S3, the processor handling the data mining will carry out filtering of the multiple data sources by the user inputs to create a subset of data sources. This part of the methodology will reduce the totality of the data sources down to those that are likely to be of interest to the user. After the subset has been defined, then the next step is the step S4 of selecting a ruleset according to the first user input defining the business type. The ruleset will define one or more rules that are to be used in data extraction. This is carried out in step S5, which constitutes the applying of the selected ruleset to each data source of the subset of data sources to extract information relating to the data source.

Once this stage of the process has been completed, then the final step, step S6, is the step of presenting an output comprising at least some of the data sources of the subset of data sources and respective extracted information. This presentation will normally be by displaying information to the user, but could take the form of the creation of data file which could then be saved for later retrieval. Likewise, the presentation could take the form of printing out a document for the user to review.

The nature of the process is that a subset of data sources is created according to the user inputs. The service provided to the user can also support the indexing and free text searching across this!!specified group!! of websites.

Whilst full text indexing is well understood and dominant in internet search engines, these engines provide results based upon their entire portfolio, or in best case, moderated by a place name. However, the combination of search processing and the graphical user interface allows search results to be specific to the selected websites. In one embodiment, this could work as follows.

The user selects the dataset to be displayed (i.e. all restaurants in the UK). The dataset is displayed geographically on a map, each data item being represented by a pointer on the map. The user selects the websites by "marking" the pointers in some manner (typically by using a mouse to enclose the pointer in a rectangle). The user enters a free text search. This can encompass a list of words, linked by Boolean logical operators such and AND, OR, NOR and other logical operators such as NEAR. The search criteria and list of selected websites are sent to the Search Processing Module 1 0. The Search Processing Module 1 0 submits the search to the Index datastore 9.

The Index returns a list of all the web pages that match the search criteria. The Search processing Module 10 compares the matched web pages against the selected websites. The matches that originate from the selected websites are returned as results to the user.

One example of an output that could be provided to the user is shown in Figure 4. This shows an output that has been generated by a search on "restaurants", as business type, and "NN" as the geographic area. The user input "NN" is the first two letters of a UK postcode, and effectively refers to the town of Northampton. The data sources are shown in the output, listed as "Site 1" to "Site 25" in column one. Further columns display extracted information shown for each of the respective data source (each of which is a website). The extracted information can be the number of times that a word appears on the website, or could comprise more complicated processing. For example, in the ruleset for restaurants could be included a rule that searched for wine prices, and generates an average price for the wines at the specific data source.

The ruleset used to extract the information about each data source is specific to the business type of "restaurant", and produces information that can be used by the person who has made the query. For example, one piece of information extracted by the ruleset is the number of times that the word "vegetarian" appears on the websites. This could be used, for example, to find those restaurants that could be interested in buying vegetables, Other data, such as data about wine lists could indicate the potential status of the specific business. In this case, a higher average wine price could indicate a more upmarket restaurant that is less price sensitive in respect of purchasing.

Claims

CLAIMS1 A method of analysing data sources comprising: * accessing multiple data sources, * receiving a first user input defining a business type, * receiving a second user input defining a geographic area, * filtering the multiple data sources by the user inputs to create a subset of data sources, * selecting a ruleset according to the first user input defining the io business type, * applying the selected ruleset to each data source of the subset of data sources to extract information relating to the data source, and * presenting an output comprising at least some of the data is sources of the subset of data sources and respective extracted information.
2. A method according to claim 1, and further comprising receiving one or more further user inputs defining one or more further features for filtering the multiple data sources.
3. A method according to claim 1 or 2, and further comprising adapting the selected ruleset according to a received user input, prior to applying the selected ruleset to each data source of the subset of data sources.
4. A method according to claim 1, 2 or 3, and further comprising receiving a user input defining a text query, searching the subset of data sources for the text query, and presenting an output comprising the results of the search.
5. A method according to any preceding claim, and further comprising applying a standard ruleset to each data source of the subset of data sources to extract information relating to the data source.
6. A system for analysing data sources comprising a processor arranged to: * access multiple data sources, * receive a first user input defining a business type, * receive a second user input defining a geographic area, * filter the multiple data sources by the user inputs to create a subset of data sources, * select a ruleset according to the first user input defining the business type, * apply the selected ruleset to each data source of the subset of is data sources to extract information relating to the data source, and * present an output comprising at least some of the data sources of the subset of data sources and respective extracted information.
7. A system according to claim 6, wherein the processor is further arranged to receiving one or more further user inputs defining one or more further features for filtering the multiple data sources.
8. A system according to claim 6 or 7, wherein the processor is further arranged to adapt the selected ruleset according to a received user input, prior to applying the selected ruleset to each data source of the subset of data sources.
9. A system according to claim 6, 7 or 8, wherein the processor is further arranged to receive a user input defining a text query, to search the subset of data sources for the text query, and to present an output comprising the results of the search.
10. A system according to any one of claims 6 to 9, wherein the processor is further arranged to apply a standard ruleset to each data source of the subset of data sources to extract information relating to the data source.
11. A computer program product on a computer readable medium for analysing data sources, the product comprising instructions for: * accessing multiple data sources, * receiving a first user input defining a business type, * receiving a second user input defining a geographic area, * filtering the multiple data sources by the user inputs to create a subset of data sources, * selecting a ruleset according to the first user input defining the business type, * applying the selected ruleset to each data source of the subset of data sources to extract information relating to the data source, and * presenting an output comprising at least some of the data sources of the subset of data sources and respective extracted information.
12. A computer program product according to claim 11, and further comprising instructions for receiving one or more further user inputs defining one or more further features for filtering the multiple data sources.
13. A computer program product according to claim 11 or 12, and further comprising instructions for adapting the selected ruleset according to a received user input, prior to applying the selected ruleset to each data source of the subset of data sources.
14. A computer program product according to claim 11, 12 or 13, and further comprising instructions for receiving a user input defining a text query, searching the subset of data sources for the text query, and presenting an output comprising the results of the search.
15. A computer program product according to any one of claims 11 to 14, and further comprising instructions for applying a standard ruleset to each data source of the subset of data sources to extract information relating to the data source.