WO2001035274A1

WO2001035274A1 - More efficient database research system

Info

Publication number: WO2001035274A1
Application number: PCT/US2000/030786
Authority: WO
Inventors: Edward J. Walters; Philip J. Rosenthal
Original assignee: Walters Edward J; Rosenthal Philip J
Priority date: 1999-11-10
Filing date: 2000-11-09
Publication date: 2001-05-17
Also published as: WO2001035274A8; CA2390701A1; AU1591401A

Abstract

A method for searching, sorting, and displaying database search results to identify and highlight important or authoritative records. The invention is applied to legal research, permitting users to find authoritative documents that are missed by standard search mechanisms and to sort search results to more efficiently find the most important records. An apparatus and method employing a citation analysis algorithm to identify authoritative records. The apparatus and method simultaneously display the full text of a record, as well as the complete search result list in a side panel, allowing the user to resort to search results on the fly. The apparatus empowers users to sort search results in a variety of ways, either before the search or within the list of results, to expedite review. The apparatus also identifies records from prior search results and that have been reviewed previously by marking them as 'viewed' links.

Description

MORE EFFICIENT DATABASE RESEARCH SYSTEM

RELATED APPLICATIONS

This application claims priority under 35 U.S.C. § 119 to provisional application No. 60/164,549, filed November 1 1, 1999, the entirety of which is incorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates generally to the field of searching databases, sorting and displaying results, and delivering records to users. More particularly, the present invention relates to an apparatus and method of expediting the review of records responsive to such a search, by more efficiently displaying, representing, sorting, and navigating such responsive records.

RELATED ART

For centuries, researchers have pored over books, reading document after document to inform their knowledge of certain fields. This kind of research is both time- consuming and cumbersome. Researchers frequently have to review documents that are not on point, and often fail to identify or locate relevant documents. With the advent of computers, companies have created voluminous searchable databases of research materials. Research materials can comprise files in various formats, from unstructured strings of characters, sentences, or text files, to very highly structured data. They can be of a wide variety of data classes, such as words, numbers, graphics, etc.

In a computerized search, the user enters a search query, usually using "keywords" or Boolean search terms, and the computer system responds by presenting a list of documents in the database that meet the requirements of the search. The user can then review responsive documents, search within that subset of responsive documents, or conduct another query. Research of this sort generally takes place on a local computer system, on compact discs or other storage devices, over a dial-up modem connection, and more recently via the Internet. One great advantage of searching αatabases by computer is that the user ma determine how broadly or narrowly to conduct text searches. Thus, to a certain extent, the user can control the number of documents returned in response to a query. This is especially helpful because queries often return hundreds, or even thousands, of responsive documents. To be thorough, researchers frequently must review each and every one of these documents. One example of this type of text retrieval system is the "Lexis/Nexis'^' system operated by Anglo-Dutch conglomerate Reed-Elsevier.

Despite great advances in computer-assisted research, reviewing documents responsive to a search query remains time consuming, inefficient, and dreadfully dull. Searches yield sets of responsive documents that are both underinclusive and overinclusive. All too often, queries fail to return relevant documents because they are not literally responsive to a user^'s Boolean search request. Similarly, Boolean searches return irrelevant documents because they are merely incidentally responsive. In order to tell whether a query was well formulated, researchers often must review scores of responsive documents one document at a time. The user may be required to review every document in a long list in order to guarantee the comprehensiveness of the research assignment. Many times, reviewing documents responsive to a query is akin to looking for a needle in a haystack.

Leaders in the computerized research industry, such as Westlaw and Lexis/Nexis, usually sort search results in a fixed hierarchy, by order of authority then by date. For example, documents from the most authoritative sources are displayed first, and within each source, the documents are sorted in reverse chronological order. Other research systems sort search results according to "relevance," which typically means displaying documents that contain the most instances of the search terms first. Users of such databases generally pay for such searching in one of four ways: (1) pay a fixed-rate subscription for access to all or a set of the materials; (2) pay for discrete events, such as a database search, or for printing a document; (3) pay for time logged into the database; or (4) some combination of the above. This leads to a great deal of account information for each user of the database - in addition to billing information, many services maintain a record of transactions, such as queries run on the database, documents printed or downloaded, or ongoing search requests. Unfortunately, research systems of today often fail to identify the most relevant documents because they rely on very simple and sometimes highly inappropriate counting algorithms to determine whether and to what extent a document is relevant. These systems simply count, for example, the number of times a certain word or group of words appears in a document and simply display the results according to count. But the number of times a word or a group of words appears in a document alone often does not determine the document's relevance. So research systems that rely on simple counting algorithms to determine relevance frequently fail to recognize and identify some of the most important, influential, popular, or authoritative records in a database or a group of databases. Thus, the quality and comprehensiveness of the research is seriously undermined.

A further disadvantage with typical computerized research systems is that search engines will return a list of only the titles or other identifiers of responsive records, which tells very little about the contents of those records. To view in full a record in the list, the user must use a mouse or other pointing device to click on a link in the list, which takes the user to the beginning of that individual record. The user can either skip sequentially from record to record, through the entire list of records, in the order they were listed, or jump back and forth between individual records and a list of identifiers. For a user to skip several records ahead or back in the list, she must either review all the records in between, or return to the list of identifiers and click a separate link. Review of results involves a lot of toggling back and forth between a list of responsive records and the full body of the records themselves, and especially with slow connections to the Internet, this can be time consuming.

One further disadvantage of current computerized research systems is that users frequently retrieve the same records, over and over, in subsequent searches. One common example is that a user conducts a search over the database of research materials, yielding 80 results. The user reads all 80 records and decides that she did not find a record responsive to her needs. The user then reformulates the query, to which the search engine returns 50 responsive records. Unable to remember all the titles of the 80 records already reviewed, the user must then read through all 50 records returned by the second query, even records she has already reviewed. Another disadvantage of the current system is that the review of account information remains either inaccessible, hidden, or widely disaggregated. Clients of such research services, especially in fee-for-service or fee-for-time transactions, seldom know their billing totals until the end of the billing cycle. Similarly, previously run search requests and document delivery information are sometimes available only by calling the company by telephone.

Accordingly, there is a need recognized by inventors in several industries, including but not limited to legal research, for an apparatus and method for expediting computerized research by providing users more information about their search queries, search results, and their own usage and account information.

SUMMARY OF THE INVENTION

In view of the problems associated with digital search and retrieval systems, one object of the present invention is to expedite computerized research by providing an apparatus and method for displaying, representing, sorting, and navigating responsive records.

Another object of the present invention is to provide a mechanism for displaying a list of records responsive to a user's search that includes user-defined portions of responsive records. Providing more information in a list of responsive records reduces the need to access the full text of records in order to determine whether the record is relevant to the user's research.

Another object of the invention is to reduce the user's need to switch back and forth between lists of identifiers of responsive records and the full body of those records, by displaying fields or other portions of the records in the list of responsive records along with the record identifiers, and by displaying the list of responsive records along with the display of any responsive record in full.

Another object of the invention is to expedite review of responsive records, both by allowing users to sort the list of such records, and by showing users which records they have already reviewed. This may also be achieved by listing responsive records in a side panel while the user reviews the full text of any responsive record, allowing the user to jump forward or back a number of records at a time. This may also be achieved by allowing users to re-sort the list of responsive records in said side pane. In two other aspects of the present invention, an apparatus and a method for expediting legal research in computerized searches of legal materials, including without limitation judicial opinions, statutes, regulations, regulatory decisions, SEC filings, briefs, pleadings, docket entries, treatises, articles, and other law-related information, are provided.

In yet another aspect of the present invention, information retrieved from searches over databases of subsequent history information for such legal materials is displayed in user-defined lists. This embodiment includes displaying a list of document titles in a side pane while viewing the full text of any document returned by the search. In a further aspect of the present invention, a list of the titles of responsive records, as well as user-defined portions those records, are displayed in a list of search results.

Typically, although not necessarily, these and other objects of the present invention are implemented along with or as part of a computerized information search and retrieval system. The user conducts a search by selecting a database and parameters for the search (including without limitation date or field restrictions), then enters a Boolean or other search query. The user then selects display preferences, such as how she wants to sort the responsive records and what portions of those records (if any) should be displayed in the list. The computer system performs a search of a database and/or associated alphabetical concordance and returns a list of records that are responsive to the user's search.

The system then displays a list of responsive records according to the user's preferences. For the preferred embodiment of a display of a list of responsive legal documents, the list may include any combination of the following elements: a descriptive title of each document, its docket number, citation, source, date information, a search relevance ranking, citation ranking, combinations of rankings, as well as any text from the document the user elected to display. Displayed text could include, by way of example and not limitation, the first paragraph of the document, the first 50 words of the document, the first n lines of the document, n an integer, the paragraph of the document most relevant to the search, n words surrounding the most relevant appearance of search terms, n an integer, or no text at all.

In a preferred embodiment, relevance is computed as a function of the frequency of appearance of the search terms and their proximity. For example, in one preferred embodiment, records that have a higher frequency of search terms are considered more relevant than those with a lower frequency . In another preferred embodiment, extra weight is given to records that contain all or most of the search terms, even if the absolute frequency is lower. In a further preferred embodiment, the number of words between each search term and the nearest search term is computed. Records that have a higher frequency of pairs of search terms with few words between them are deemed more relevant that those with lesser proximity between them. In another preferred embodiment, frequency and proximity are combined.

For example, assume there are three documents, 1, 2, and 3, found by a Boolean search for three search terms, A, B, and C. Assume the documents have the following frequencies and proximities:

Document 1 : A at positions 2, 76, 756, 767, 4956, and 95654.

Document 2: A at 2 and 6, B at 3 and 9, C at 17

Document 3: A at 2 B at 877 and C at 8604

If simple frequency is the measure of relevance, document 1 is the most relevant with six search terms, then documents 2 and 3. If norming mechanisms such as "double weighting" are used for occurrences of all three search terms, 2 is the most relevant, with 1 and 3 tied for second most relevant. Using proximity, document 2 is the most relevant because the first A and the first B are adjacent, the second A and second B are 3 words apart, and the last B and first C are 8 apart. If an index is created by summing the proximity of the nearest three pairs, the index for document 3 is 12. For document 1, the index is 775 and for document 3 there are not enough pairs to evaluate proximity. In order of relevance, the documents are sorted 2, 1, 3. Frequency and proximity may be combined in a variety of ways. For example, they can be both applied separately and the results averaged. Alternately, the proximity index can be normalized to the greatest frequency of search terms found in any record and then added to the frequency.

The user could scroll up and down the list, reviewing both the titles or other identifiers of responsive records and relevant text portions of those documents. The user could resort the list on the fly, by any information in the list. She could also change the amount or type of text, fields, or other portions of records displayed in the list. This allows the user, at a glance and without accessing the documents in full, to better evaluate the usefulness of her query, and to evaluate which records are important to the research task at hand more efficiently.

The display also may allow the user to determine which records, if any, have been returned by prior searches or that have been selected by the user, for example, for printing. To do this, a running list of records, identified by their unique identifier, is preserved and associated with the user's unique ID. When a subsequent set of records is to be displayed, the identifiers of those records are checked against the preserved list to find the repeated documents. Repeated records are identified in the display, for example, by making their identifiers bold or of a different color. Similarly, if the user selects the document, for example by clicking on its identifier or by printing it, that document's identifier is added to a different list which is also associated with the user's ID. The identifier's of new records to be displayed are also checked against this list to determine those already selected. They are displayed in a different fashion, for example by using a different color, boldness, or a special icon. Typically, the user selects any document on the list, using input devices such as a mouse, keyboard, stylus, microphone, or other selection device. For the preferred embodiment where the system or method of displaying records is combined with a search to first identify those records, he computer system then displays the full record with search terms highlighted therein. The user can jump from term to term, page to page, or field to field within a record, or jump from one record to the next using, for example, navigation keys. In addition, the system displays a list of the titles of all responsive records in a side panel. This list may be re-sorted on the fly, and the user may scroll up and down the list of responsive records, independent of the individual record being displayed. Using the methods described above, the user can select any record in the side panel — including records several places ahead or behind the displayed record in the complete list - and the system will display the selected record in full, with the complete list of responsive records in a side panel.

In one preferred embodiment of the present invention, the database is structured using XML tagging. This permits ready identification of all fields and other unique portions of each record. XML-identified content is then used in the displays in a variety of ways, including to identify the records. Another preferred embodiment of the invention is to apply the system to databases of judicial opinions, statutes, regulations, administrative opinions, and other legal documents. This embodiment is illustrated in the drawings.

The present invention is further directed to a method for identifying, sorting and displaying records that are important to a user^'s search request. The method comprises the steps of: (i) creating a look-up table, which is an organized concordance of all elements in a database (including without limitation: records, fields, words, numbers, citations, illustrations, and the like), said look-up table to include information describing each element in the database; (ii) entering a search query for the database, including preferences about how the results should be sorted, such as by popularity, authoritativeness within the database, or authoritativeness among responsive records; (iii) searching the database or set of databases for records based on the user^'s criteria; (iv) comparing the records returned by the search to the entries for those records in the look-up table; (v) sorting records returned by the search according to information in the look-up table; (vi) displaying a list of records responsive to the query, sorted according to the user's preference.

In one preferred embodiment of the invention, records are sorted according to their authoritativeness within the database. A look-up table is created that lists, for every record in the database, all references to the record in question. For example, if record number 10 were cited three times in the database, the lookup table read: 10: 3. In a further preferred embodiment, the look-up table lists the number and/or the location in the database of each such reference to the record in question. In the above example, if record number 10 were cited three times, in document 2 at character 56, record 20 at character 56, and in record 83 at character 182, the table could contain the entry 10: 3 :: 2(56), 20(345), 83 (182). The search results are sorted using the total number of references to the record, so that the records referenced most frequently are displayed first in the list of responsive cases. For example, if a database had 100 records and the search of step (iii) returned 3 records, the algorithm would locate the three records in the look-up table, identify the table entry corresponding with the total number of references made to each document, compare the entries, and display a list of the records sorted by to that number. If the first record were referenced 4 times, the second 24 times, and the third 8 times, the search results would be sorted record number 2, then 3, then 1. In another embodiment of the present invention, records are sorted according to their authoritativeness within the set of responsive records only. In this embodiment, the database is searched as in steps (i) - (iv) above, returning a set of records responsive to the search and identifying those records in the look-up table. Instead of using the total number of references to the list of responsive records, the algorithm would read down the list of all references, but only count references within documents returned by the query and sort the responsive records accordingly.

For example, if a query returns 5 responsive records, records number 10, 20, 30, 40, and 50, the algorithm would locate record number 10 in the look-up table, then review the list of all references to document 10 in the database. If the total number of references were 3, that entry might look as follows: 10: 3 :: 2(56), 20(345), 83 (182). In the present embodiment, however, only one reference would be counted, the reference in document 20 at character number 345, because of the referencing records (records 2, 20, and 83), only record 20 was originally returned by the search. The algorithm would then repeat this process for records 20, 30, 40, and 50. Because records returned by the search will generally share a tighter topical nexus than records in the database generally, this "closed loop" relevance algorithm counts references with a greater probability of being germane to the research task at hand, factoring in the quality of the reference instead of the raw quantity. In yet another aspect of the present invention, an additional measure is used to identify germane references to records returned in a search, namely only counting references withm a specified proximity of one of the search terms. For example, with a text database searched for particular terms, references would only be counted if they came within n words, n an integer, of any of the search terms. For example, assume a text database contains 50 documents and is searched for the word "cow" and further assume that 7 documents are returned by the search. References to the 7 documents would only be counted iζ for example, they were within 25 words of an appearance of "cow" in each referencing document. Sorting would then occur as before.

In another further aspect of the invention, documents not literally responsive to the search query, but nonetheless relevant to the research task, are identified. Where the algorithm determines that a certain number or percentage of records reference a document not returned by the search, the algorithm identifies that document for the user. In this embodiment, the look-up table of step (i) includes, for each record, a list of each reference that record makes to all other records in the database. In the example above, if record 10 is cited three times in the database, and record 10 itself cites two other records, the look-up table entry could read: 10: 3 :: 2(56), 20(345), 83 (182) :: 45(8643), 58 (4003). When the user conducts a search, the algorithm counts the number of references that the responsive records make to other records in the database. If this number of references for any record in the database that is not in the search result exceeds a certain threshold, that record is identified as another important or authoritative record that was missed by the original search.

For example, assume a search of 100 records that returns five responsive records: 10, 20, 30, 40, and 50. Suppose further that the look-up table reveals that four of these five records cite record number 83. If the threshold were set at three references, or using an alternative threshold, citation by 50% of the responsive records, then record number 83 would be identified as potentially germane to the search.

In another aspect of the invention, additional authoritative or important records are identified by the algorithm described in the preceding paragraph, with the modification that references to other records in the database or set of databases are counted only if those references fell within a specified proximity of the characters, words, or features that were identified in the record which resulted in its inclusion in the search result. In a further aspect of the invention, the algorithm ranks records according to their popularity. In this aspect, the look-up table in (i) includes, for each record in the database, information about the number of times that record has been delivered to users of the system, including but not limited to page views, print requests, faxes, or downloads. For example, if record 10 had been printed 456 times, its entry in the look-up table could read 10: 3 :: 2(56), 20(345), 83 (182) :: 456. Indexing and searching would be conducted as in steps (i)-(iv) above, then the algorithm would compare the number of deliveries of the record and sort the documents in order of popularity, thus computed.

In another aspect of the invention, all references in all records in the database or set of databases are identified in Extensible Markup Language (XML) for easy identification for use with any embodiment of the invention. In another aspect of the invention, all references in all records in the database or set of databases are identified in hypertext markup language (HTML) for easy identification for use with any embodiment of the invention. In another aspect of the invention, separate look-up tables are constructed for different types of information collected about each record. In a separate embodiment, all of the information is combined in a single look-up table. In a further embodiment, look-up tables are not constructed at all, and all information in the look-up table of (i) is calculated "on the fly" by a separate search of the entire database or subsets therein. In a further embodiment, the system uses one or more three-dimensional look-up tables.

The present invention is yet further directed to a method or process for assembling and displaying account information for the user of a search engine. The method includes collecting information from a number of different back-office systems and displaying them in a single, flexible utility. The utility displays a number of different headings or "tabs," each of which offers the user access to a different kind of information about the user's account.

In one preferred embodiment of the invention, the utility assembles the user's billing information and displays it to the user on demand. The utility displays invoicing information, such as the user's identifying account number, client and matter numbers, the date and amount of the last payment, the amount and date of any pending bill, and an accounting of all billable events since the last invoice. The list of billable events since the last invoice could contain information on the amount of time spent on the system; the date, number, type, and time of fee-based transactions conducted on the site, the total charge for said services, or an accounting of any other item charged to the account since the last invoice. The information could be displayed as quickly as it was processed by back-office or accounting software, up to and including in real time.

In another further embodiment of the present invention, the utility collects and displays account information sorted by the user's internal client and matter billing numbers. This allows a user to display all transactions she chose to bill to a particular client. The log contains information such as the identifying name of a document delivered for a fee, identifying citation information, the source of the document, the date it was delivered, and the price and method of the delivery, and the client billed.

Another embodiment of the invention allows the user to export this billing data to a user's internal billing system so that the user automatically can generate a single bill that includes fees for professional services and charges to be passed through to the client. In another further aspect of the invention, the utility tracks information about the current research session in real time. Such information includes the exact amount of time on the system, the client to whom the research is to be billed, and whether the user is being billed on a time, transaction, or subscription basis. The user could change the client number to be billed using this utility. A further embodiment of the invention allows the user to select from a list of previously entered client names and associated billing numbers.

In another embodiment of the present invention, the utility collects and displays information about the user^'s activity within the database. The invention collects such information as past search queries, the date on which they were run, the number of documents responsive to the query, the client billed for the research, or other identifying information. In a further embodiment of the invention, this information is displayed as a hyperlink, or with an accompanying icon, by which the user may re-query the database using the same criteria. In still another embodiment of the present invention, the utility collects and displays information about documents the user has selected for delivery, whether through printing, e-mailing, downloading, faxing, mailing, or any other delivery mechanism. The system displays a list of documents, including information such as identifying citation information, the source of the document, the date it was delivered, and the price and method of delivery. A further embodiment includes either a hyperlink, icon, or other link by which the user could access any document from the list.

A further embodiment of the invention lists the amount of time spent on the system on certain dates, and the amount billed for each. A still further embodiment of the invention calculates and displays for subscription users what percentage of fee-based transactions or time was allocated to which clients, allowing the user equitably to apportion a subscription fee among active clients. Another embodiment of the invention would display the costs of transactions and compare the cost of the same transaction on other systems.

In another aspect of the invention, the system keeps a running tally of search queries the user wishes to continue running periodically. The utility tracks information such as the search query, databases to be queried, restrictions on the search, the date it was entered in the system, the frequency of the query, a list of all occurrences of new document entries for each search, and the client billed. A further embodiment of the present invention includes hyperlinks or icons, allowing the user to re-query the system on demand.

In a further aspect of the invention, the utility tracks password and encryption settings for the user. This allows the user to change her password, change the settings for which searches and fee-services should be logged and which should not, and to change the level or type of encryption for the connection.

The steps in all methods may be performed in any order, unless expressly stated otherwise.

FEATURES AND ADVANTAGES OF THE PRESENT INVENTION

It is a feature of the present invention that users can sort search results based on a number of different criteria, including without limitation authoritativeness generally, authoritativeness within a subset of records, popularity, or novelty.

It is a further feature of the present invention that one can distinguish records that are authoritative or important for a particular issue from records that are generally important. This is accomplished by only counting references from within a search result, because these records are pre-selected by the original search to deal with that issue. Demanding that references have a certain proximity to elements used in the original search further enhances this feature of the invention. It is a further feature of the present invention that one can sort records by their popularity. This sorting feature allows the system to "learn" about records in the database from usage patterns and recommend records in subsequent searches.

It is yet another feature of the present invention that the authoritative or important record can be found very quickly because a prior look-up table was constructed, obviating the need for a search of an entire database. It is a further feature of the present invention that these records and references may be identified quickly because of the use of

HTML, SGML, and/or XML tagging.

It is an advantage of the present invention that it allows additional important or authoritative records to be identified that were missed by the original search of the database or databases. Search queries often are constructed imperfectly, excluding records germane to the research task. The present invention identifies these records quickly where they might only otherwise be found by comparing the references of every responsive record.

It is a feature of the present invention that the system displays user-defined text in a list of records responsive to the user's search. This feature is an advantage because it better allows users to tell whether any given record in the list is germane to a research task, without having to look at the full record, saving time, steps, computing function, and effort.

It is a further feature of the present invention that the user may re-sort a list of responsive records on the fly, without having to conduct another search. This allows the user to find the most relevant documents quickly, using a number of different techniques. It is a further feature of the present invention that the list of responsive records identifies documents the user has recently viewed. This is an advantage because it reduces the duplication of effort involved with reading the same document twice. This feature is also an advantage because it allows users more quickly to review a list of records and to compare results of one search against the results of a previous search.

It is a further feature of the present invention that the system displays a sortable list of search results in a side panel when the user is viewing a complete record from the list. This allows the user to jump several records ahead in the list, to know whether she has previously viewed records in the list, or to re-sort results, without going to a separate list of responsive records.

These two features provide an additional advantage to the system, allowing the user to access simultaneously the text of a record and the list of all responsive records, thus saving the transaction time of jumping back and forth from one list to another. Especially for remote transactions over a computer network or the Internet, this feature reduces the total number of transactions, saving time and adding stabibty to the research session.

It is a feature of the present invention that users of research utilities may have instant access to a number of different types of account information, many of which they currently have no access to at all. It is a further feature of the present invention that the user can access billing information for past and pending bills, as well as information since the last invoice. This allows consumers to meter their usage and to better control costs, as well as allow self- auditing of research habits in order to save money. A further feature allows the user to view billing information for the current session. This offers the advantage that users can monitor costs more closely and stop researching when the cost becomes too high. Current fee-for-service or fee-for-time services offer no such feature, so by the time the expensive research bill arrives at the end of the month, the user has already incurred the cost.

It is a further feature of the present invention that users can bill research on the system to different clients, allowing the user to "pass through" some of the costs of research. The user may at any time access itemized client billing information, which facilitates users billing clients for research costs. The system stores client numbers, an advantage because the user need not re-enter client numbers for each transaction, but instead may select from a list. It is another advantage of the present invention that it allows subscription-based users to apportion their bill among clients for whom they conduct research. Another advantage of the system is the ability to export that data to accounting systems, to reduce the amount of time required in data entry. It is yet another feature of the present invention that the user can view usage information, such as past queries, past document deliveries, pending search requests, and other uses of the system. Another feature allows the user viewing such information to recall a previously viewed document or re-query the database. This has a number of advantages. It facilitates research conducted over a number of days, research interrupted by disconnection or network failure, or follow-on research. It also allows the recall of documents already accessed or delivered, but later misplaced. It is yet another advantage of the present invention that it allows users to resume interrupted research or to conduct research sessions over several days.

Additional features and advantages of the present invention are set forth in part in the description that follows, and in part are apparent from the description, or may be learned by practice of the invention. The features and advantages of the invention may also be realized and attained by means of the instrumentalities and combinations particularly set out in the appended claims.

BRIEF DESCRIPTION OF THE FIGURES

The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which: FIG. 1 illustrates an exemplar}' embodiment of a window displayed on a CRT or other display, which can be used as a user interface for the present invention, including a search dialog box.

FIG. 2 illustrates a list of records returned by the user's search, including several sorting, display, and navigation features.

FIG. 3 illustrates the display of the full text of a responsive record, along with a side panel including the list of titles of the returned records.

FIG. 4 illustrates a flow diagram for a query in one embodiment of the invention. FIG. 5 illustrates a flow diagram of options provided to a user by one embodiment of the invention after a query is performed.

FIG. 6 illustrates a flow diagram of one embodiment of the invention after responsive records are displayed in part or in full.

FIG. 7 depicts a flow diagram illustrating one embodiment of the present invention.

FIG. 8 depicts a flow diagram illustrating an alternative embodiment of the present invention in which a citation look-up table is used.

FIG. 9 depicts a flow diagram illustrating another embodiment of the present invention which tracks user experience data, preferences and associations. FIG. 10 depicts yet another flow diagram illustrating various ways of sorting lists of responsive documents according to the present invention.

FIG. 11 depicts a flow diagram illustrating a preferred embodiment for identifying previously unidentified documents.

DETAILED DESCRIPTION

An apparatus and method are described for displaying records. In the preferred embodiment, such apparatus and method are combined with an apparatus and method for expediting legal research by displaying, representing, sorting, and navigating text files identified by a search engine. In the following description, for the purposes of explanation, numerous specific details such as mathematical formulae, algorithms, menus, and the like are set forth in order to provide the best mode of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without these specific details, and is not limited to the specific details shown and described. In other instances, well known structures and devices are shown in block diagram form to more clearly set forth the present invention.

FIG.l shows a search screen or window 100, as may be displayed on a CRT, as an example of one possible user interface for the query screen on a research system. A user first selects a database or group of databases on which to conduct a search 103, for example, cases of the U.S. Supreme Court. The user then delimits the search using such factors as date restrictions 104, and then enters either a Boolean text search 106, a field search within the database 105, or a combination of the two.

Before or after conducting the search, the user may customize the list of records that will be returned, including how the results of the search will be organized and what information the system will return about each record. She may select fields by which to sort results on the search screen 100, such as the date of the record, the relevance of the record to the search, or by which database the record came from 101. The user may sort first by one criteria, then break ties using second- and third-order sorting fields. So, for example, if the user chose to sort by "Court Hierarchy," then by "Date," then by "Search Relevance," the system would display documents issued by the highest court first, for example from the U.S. Supreme Court. Supreme Court documents would be further sorted in date order, and cases from the same date would be listed in order of relevance. In the preferred embodiment, instead of customizing the sort order, a user could also use the system default.

The system also allows the user to decide how many records to list per page in the list of returned records 107. The user may select a smaller number of records to display on each web page, for example, to speed the loading time of the web page. Conversely, the user may select a large number so that she can view all the returned records without having to link to another page. Finally, the user may customize what kind of text or other portions of the record will be displayed in the list of returned records 102. For example, with text documents, the user may select the first 6 lines of the document, the first 12 lines of the document, the first paragraph of the document, the most relevant passage or paragraph from the document, or no text at all. In one embodiment, the user then conducts the query by selecting "Search" 107.

In one preferred embodiment of the invention, it is combined with a search engine that searches the selected database and returns a user-customized screen 200 listing responsive records, which is illustrated in FIG. 2. For each responsive record, the system lists, for example, a document title or other identifier 210, which is hyperlinked to the full record . Each entry also may display information that identifies the record, its source, date, and a relevance score for the query, as well as selected text or other portions of the record itself 201. Records previously viewed by the user would be identified as such, for example, either through the use of text, icons, or display of the record identifier as a "viewed hyperlink," using, for example, either a different color for the identifier, or the browser default for viewed hyperlinks. In a preferred embodiment, records that appeared in a previous list of responsive records, but which were not selected for a full-text view, are identified as such, either with text, an icon, or by the use of a different color for the hyperlinked identifier. Thus the user can avoid duplication of effort, by choosing not to view records in full that she has already reviewed. She would also be able to identify records added to the list after she modified a search query or conducted a new search with different terms.

As requested by the user, the system will display a certain number of records per page 202, as well as a link the user can use to jump to the next n records in the list 203. This display allows the user to scroll through the list of records in whatever order is most helpful in the search. The display of text or other portions of each record reduces the amount of time and effort spent by the user switching back and forth between screens trying to discern whether a given document is important to the research being conducted. Because the user can customize and view text or other portions of records in the list, she has more information on the list screen, allowing her to avoid reading records that are demonstrably unimportant or demonstrably irrelevant, based on a reading of the displayed portion.

In one preferred embodiment, the user can re-sort the document list on the fly, either by clicking any of the headers 204, in this example "Case," "Court," "Date," or "Relevance." Clicking "Date," for example, would re-sort the records in descending order of the record's issue date. The user could also re-sort the list using the sorting options in the side panel 205, which offers the same multi- level sorting options as the search query screen 101. From the list of displayed cases in FIG. 2, the user can also change the number of records displayed per page 206 and the amount and type of text or other portions of the record to display in the list 207, the same options offered in the search query screen in 107 and 102. When the user has selected new display properties, she presses "Resort" 208, and the system re-sorts the list. These sorting and re-sorting options allow the user to review the list of responsive records more efficiently—especially when the list is long, or when the user is looking for a specific document. Re-sorting allows the user to manipulate the list to bring important records to the top, using any of a number of different sorting algorithms. The user can conduct a new search ("N"), modify the current search query ("M"), or search within the displayed results ("R") by clicking the appropriate button 209 for those tasks. The user may also decide to sort the results by other sorting mechanisms, such as the extended reference search method described above by clicking similar buttons or choosing different options in a tab. In order to view any record in the list in full (or to view an expanded portion of any record), the user would click the hyperlinked identifier of the record. The system would then display the record in full, as illustrated in FIG. 3. One section of the screen would display the full record itself 301. The user can navigate the text using a standard browser scroll bar, by using a keyboard to move the cursor within the record, or by way of navigation buttons 305-307. By using "Term" navigation buttons 305, the user can jump from search term to search term within the document 305. Similarly, by using "Page" navigation buttons 306, the user can jump ahead or back one page at a time. Finally, the user can use the "Case" or "Document" navigation buttons 307 to jump one record forward or back in the list of responsive records. During this process, no matter what part of the record the user is viewing, the record's title or other identifier and citation information appear in a navigation bar 308 at the bottom of the screen.

A section of the display screen, depicted as panel 302 in FIG. 3, is used to display a list of responsive records 302. The list displays in the same order as it last appeared in the record list in FIG. 2, that is, before the user selected the full text or other portion of a record on the list. The user may scroll through this list independently of the full text or other portion of a record from within the list. The title of the displayed record will appear highlighted within the list in this side panel. This allows the user to view the list of responsive records without returning to the full list, and greatly facilitates navigation of the list. For example, the user could select to view the full text of a record five titles down the list by scrolling down and selecting the hyperlinked title or other identifier of that record , without having to go back to the complete list of responsive records. This reduces the transaction time of the standard mode of reviewing records in the list. As with the responsive record list illustrated in FIG. 2, the preferred embodiment identifies records previously viewed by the user, either through the use of text, icons, or display of the title as a "viewed hyperlink," using either a different color for the title or identifier, or the browser default for viewed hyperlinks. Again, records that appeared in a previous list of responsive records, but which were not selected for a full-text view, can be identified as such, either with text, an icon, or by the use of a different color for the hyperlinked title.

The user has the option of re-sorting the list on the fly from the full record text screen of FIG. 3. By selecting the "Re-sort" tab 303, the user is given all of the same multi-level sorting options from the search query screen 101, and from the screen listing the responsive record 200. This facilitates review of recording the list, allowing the user to full the most helpful records to the top of the list, without having to go back to the full list of responsive records illustrated in FIG. 2.

Finally, from the full text view illustrated in FIG. 3, the user has a series of navigation options. She may return to the full list of responsive records illustrated in FIG. 2 by selecting a button option 304. She may also conduct a new search, modify her existing query, or search within the results by selecting one of those button options 309, similar to the options 209 in the list of responsive records illustrated in FIG. 2.

FIG. 4 is a flowchart illustrating one embodiment of the steps involved in a simple search of a database as part of this invention. First, in step 401, the user formulates and enters a query. The query may use key words or phrases and be constructed using Boolean logic. The user may then select the databases in which the search will run, for example, the U.S. Supreme Court, or the Fourth Circuit Court of Appeals (step 402). Next, the user selects the amount of text to be displayed for each responsive document in the list of responsive documents. See step 403. The user may also select the sort order for responsive documents (step 404). Documents can be sorted, for example, by court hierarchy, date, and relevance, in any order. The user may also select more advanced document sorting options. Alternatively, the user may skip steps 402, 403, and 404, instead relying on default values. The user then submits the query, and the system judges whether or not the query is well-formed (e.g., if the query is a Boolean search, whether the syntax is correct; alternatively, whether the user selected at least one database to search over) in step 405. If the query is not well-formed, the system returns an error message in step 406, then displays the query for editing in step 407. If the search is well-formed, the system queries the relevant database or look-up table associated with selected database in step 408. If one or more responsive records is not found via step 409, the query is again displayed for editing when processing is returned to step 407, perhaps with a message indicating that no responsive records were found. If responsive records do exist, the system selects text from each responsive record by default or as defined by user, as shown in step 410.

Next, the system sorts the responsive records by a default method or by a method defined by the user in step 41 1. Subsequently, the system displays the sorted list of records and, if requested by user or required by a default rule, an amount of text from each document. See step 412. The user then reviews the list (shown as step 413) and decides if the documents returned are sufficiently germane to the research topic (step 414). If the documents are germane, the user reviews the full records list in step 415. If not, the user may choose either (i) to modify the query, as shown in step 417, in which case the system displays the last query for editing and accepts editing commands, step 407, or (ii) to begin a new search, as shown in step 416, wherein the system displays a blank search page and the process begins anew in step 401.

FIG. 5 is a flowchart illustrating an embodiment of steps after a user has chosen to view a record list resulting from a search, as shown in step 415. First, in step 501 the system compares the responsive records with records that the user has previously reviewed, if any. Next, the system displays a sorted list of responsive records, clearly marking records that the user has viewed before, and user-defined text (if any). See step 502. The user then decides in step 503 if the list is sorted in a helpful way. If not, the user may, as shown in step 504, select new sort criteria from a supplied list and click page header to re-sort. The system then re-sorts the records in step 505 and displays them in an updated list (step 502).

If the method of sorting is satisfactory, the user must then decide whether or not the displayed text is useful. See step 506. If the user determines that it is not, the user selects a new text type to display in step 507. Continuing with this scenario, the system extracts the newly selected text from responsive records in step 508 and again displays the sorted list of responsive records, identifying records that the user has previously viewed, and user-defined text (if any) in step 502. If, alternately, the displayed text is acceptable, the user decides if the listed records are sufficiently germane to the research in step 509. If not, the user may start a new query as shown in step 510, or modify the current query, step 53 1 , returning to the steps in FIG. 4 in either case. The user also may consider whether the percentage of germane records is sufficiently high. If it is not, the user may return to FIG. 4 and further limit the search by conducting a query on responsive records only, as shown by step 513. If the percentage is sufficiently high, the user browses through the list of responsive records and text in step 514.

Next, the user may choose to print a record, or view a record, or both, in any order. The user may print a record by selecting it in the list in step 515. The system then displays the formatted record to the user in step 516, and the user confirms the print in step 517. Alternately, the user may choose to view a full record in step 518, in which case the system displays the full record, without special formatting, shown in step 519. This scenario is illustrated in FIG 6.

FIG. 6 illustrates an embodiment of the steps taken after a user opts to display a full record. In the main window, the system displays the selected record in step 601. The system also compares responsive records with records previously viewed by the user in step 602, and displays a sorted list of responsive records, identifying which of the records have been viewed previously (for example, by displaying their headings in different colors), and user-defined text (if any) in the side panel. See step 603. The user can interact with the system through either window. Through the main window, which displays the text of the selected record (see step 601), the user may: - Jump to the next or previous appearance of a search term (step 604).

- Jump to the next or previous record in sequence (step 605).

- Jump to the next or previous page within a record (step 606).

- Print the unformatted record (step 607).

- Select formatted printing option (step 608), causing the system to display formatted record, as shown in step 609. The user then confirms the print order which is then executed. See step 610.

From the list of responsive records in the side panel, the user may (i) decide that the listed records are not sorted in a helpful way (step 615), or (ii) decide that the listed records are not germane to research (step 611). In the first case, the user may either (i) click header to re-sort as in step 616, or (ii) select new sort criteria from the list in the side panel as in step 617.

If the user decides that the listed records are not germane to research at step 611, in one embodiment three options are presented: (i) the user may further limit search by conducting a query on responsive records (step 612); (ii) the user may modify the current query (step 613); or (iii) the user may start a new query (step 614).

The present invention has been disclosed and described herein in what is considered to be its most preferred embodiments. It should be noted that variations and equivalents may occur to those skilled in the art upon reading the present disclosure and that such variations and equivalents are intended to come within the scope of the invention and the appended claims.

FIG. 7 illustrates the general workflow process of creating the invention and a few preferred embodiments thereof. First, a database or multiple databases of digital legal content are created in step 701. This may be done by compiling electronic documents that are already in electronic format, creating electronic documents by data conversion, or any other method of data entry. Next, in a step 702, XML tags are added to the documents either manually, wherein each tag is inserted by a typist, electronically, wherein scripts are written that automatically insert tags in the proper places, step 703, or by some combination of the two. In the preferred embodiment, the tagging process is submitted to rigorous quality assurance/quality control procedures (QAQC). See step 704. Next, a look-up table or series of tables is created from the tagged documents in step 705. The system then conducts searches over the look-up table or tables, step 706, and the system then displays a list of search results or "responsive documents," step 707. The system provides for a series of sorting algorithms to make research tasks more efficient. Examples include sorting algorithms that bring certain types of documents to the top of the list, step 708, and algorithms that identify documents that are not in the set of search results, but are nonetheless germane to the research, as shown in step 709. The invention also allows the user to display the full text of any document in the list, step 710, or to check the subsequent history of any document, as shown in step 711. FIG 8. is a flowchart illustrating the steps involved in the formation of the citation look-up table. The process begins, in step 801, with a database of records that reference each other. For each record in the system, citations to other records are identified and marked, step 802. Next, each citation's "target" in other database records is identified and marked, shown in step 803. Then, in step 804, a table is created that includes three pieces of information for each record in the database: (i) information identifying the record; (ii) the number of times it cites other documents; and (iii) for each such citation the identification for the document it references and location of each citation targets.

From this data, a citation look-up table is created in step 805 in the following way. In step 806, the system reviews the entry for the first document in the table from step 804 and any other database documents it cites. The table entry for that record may or may not indicate citations to other documents in the system, step 807. If it does, the system determines whether the look-up table already includes an entry for the document that is the target of the citation. See step 808. If not, the system creates an entry in the look-up table for the document that is the target of the citation, identifying the citing document and entering 1 for the number of citing documents. See step 809. If the look-up table already includes an entry for the document, the entry for the citing document of step 807 is added and the entry for number of citing documents is increased by one (1), as in step 810. Ifj at step 807, the table entry for the record does not contain citations to other documents in the system, or once step 809 or step 810 have been completed, the system determines at step 811 whether the entry considered most recently from the table of step 804 is the last record in that table. If it is not the last record, the system goes to the next record in the table, step 812 and processing returns to step 807. If it is the last record, then the citation look-up table has been completed. See step 813.

FIG. 9 is a flowchart illustrating the steps for tracking user-experience data or user preferences and associations in the system to enhance searching and sorting, either in the same look-up table of step 913 or as a separate look-up table. These preferences may be tracked in a number of ways. In one preferred embodiment, preferences are tracked by counting the number of times a document is viewed by users or delivered to users (e.g. printed, faxed, downloaded, or delivered by some other method). Once a citation look-up table has been created (either as in FIG. 8 or otherwise) 901, two fields or "columns" must be added: one that tracks page views, another that tracks deliveries 902. The user preferences are gathered during searching, sorting, and document delivery in the system. In one such iteration, the database is queried at step 903, and a list of responsive documents is returned, step 904. From this point, the user may choose to re- sort the list of responsive documents, step 909, using any of the methods described below with reference to FIG. 4. Alternatively, the user may choose to view the full text of one of the responsive documents, step 905, or deliver a document by printing, downloading, faxing, or via another delivery method, step 907. If the user views the full text of the document, step 905, the system increments the page view counter in the look-up table entry for the viewed document. See step 906. If the system delivers the document to the user in any of the ways described herein, the system increments the print count look-up table entry for the delivered document. See step 908.

FIG. 10 is a flowchart illustrating sorting features that make research more efficient. First, at step 1001, the user conducts a search. The system returns a list of responsive documents, step 1002. This list may be re-sorted, step 1003, in a variety of ways.

In one embodiment, the list is sorted by the number of times each document has been viewed by other users (Step 1004), as described in step 906. In this case, the system consults the page view tally in the citation look-up table for each document in the list created in step 1005, and returns the list created in step 1002 sorted by this number.

In another embodiment, the list is sorted by the number of times each document has been delivered to other users (Step 1007), as described in step 908. The system consults the delivery tally in the citation look-up table for each document in the list, step 1008, and returns the list created in step 1002 sorted by this number.

In yet another embodiment, the list is sorted by the number of times that other records in the database cite to the documents in the list created by step 1002, or "authoritativeness" (Step 1009). The system determines how many times each document is cited by other documents in the database (Step 1010), and sorts the list accordingly (Step 1006).

And finally, in yet another embodiment, the list may be sorted by authoritativeness among other responsive documents as shown in step 1011. The system computes, in step 1012, the number of times each returned document in the list created by step 1002 is cited by other returned documents, and sorts the list created in step 1002 accordingly.

In a preferred embodiment, the ranking of steps 1010 and 1012 may be enhanced by including multipliers to enhance the authority of documents cited by the most authoritative institutions, such as the U.S. Supreme Court. Tne system also identifies documents that may be germane to the research task, but for whatever reason were not returned by the query to the system, step 1013. In a preferred embodiment, the system locates documents that are frequently cited by the responsive documents of list, step 1002, but are not themselves a part of the list of returned documents. This process is described and illustrated in FIG. 1 1.

FIG. 11 is a flowchart illustrating how the system identifies documents that are not literally within the scope of a search, but might nonetheless be germane to the research task. First, the user conducts a search, step 1101, and the system returns a list of responsive documents, step 1102. Each document in the list created in step 1102 cites a host of others, and step 1103 organizes information about those citations. For each document in the list, the system consults the citation look-up table of FIG. 9 and creates a new list of cited documents. See step 1103. Next, the cited documents of the list created in step 1103 are ranked, with the most frequently cited documents first, step 1104. Thus from the list of documents responsive to the user's search, the system creates a separate table of all other records cited by that original list. See step 1103.

Beginning with the first document in the list created by step 1103, the system computes, at step 1104, the significance of the number of citations. In one preferred embodiment, the system creates a "citation score" using an algorithm that divides the number of times a document in the list created by step 1103 is cited in the responsive documents of list from step 1102 by the total number of citations in the documents of list 1102 to all other documents in the system. In another preferred embodiment, the system creates a citation score using an algorithm that divides the number of documents of the list created by step 1102 that cite to a particular document in the list created by step 1103 by the number of documents in the list created by step 1102. In another preferred embodiment, the system guards against skewed citation scores using "p-norming" or other tools well known to those skilled in the art. As illustrated in FIG. 11, the system has thus resorted the list created by step 1103 by citation score order, with the most often or most authoritatively cited documents at the top. See step 1105.

Beginning with the first document in the list created by step 1105, the system determines whether the cited documents are authoritative enough to identify them to the user. In a step 1106, the system compares the citation score to a certain, pre-defined significance threshold. If the document's score exceeds the threshold (Step 1107), the document is added to a list of documents to report to the user, or a "reporting list," step 1108, and in a step 1109, the system advances to the document with the next highest citation index in the list created by step 1105. If the document's score does not exceed the threshold, processing continues to step 1110. In step 1110, if there are no documents in the reporting list, the system continues to display the list of responsive documents which have been displayed in the foreground since step 1102.

For the first document in which the significance is not above the threshold in step 1107, the system determines which documents from the reporting list created by step 1108 to bring to the user's attention. In step 1110, if there are documents in the reporting list, the system compares the reporting list created by step 1108 to the original list of responsive documents created by step 1102, removing any documents that are already part of the search result. See step 1111. The system then alerts the user that it has identified a document not part of the search result that may be germane to the research task. See step 1112.

Claims

CLAIMS What is claimed is:

1 . A method for displaying records responsive to a database query comprising the steps of:

displaying a list of identifiers for a plurality of said responsive records; and

displaying selected elements of at least one of said responsive records, wherein said list of identifiers and said selected elements are displayed simultaneously.

2. The method of claim 1, wherein said selected elements comprise the entirety of one of said responsive records.

3. The method of claim 1, wherein said identifiers comprise case citations.

4. The method of claim 1, further comprising the step of identifying and marking records displayed in their entirety in a prior search request.

5. The method of claim 1 , further comprising the step of identifying and marking records that were responsive to a prior search request.

6. The method of claim 1 , further comprising the step of sorting said responsive records.

7. The method of claim 1, wherein said responsive records include

a record name;

a record citation;

a record date; and

a record author.

8. The method of claim 6, further comprising the step of computing the relevance of said responsive records.

9. The method of claim 7, further comprising the step of computing the relevance of said records.

10. The method of claim 7, wherein said list of responsive records is sorted according to said record name.

1 1. The method of claim 7, wherein said list of responsive records is sorted according to said record citation.

12. The method of claim 7, wherein said list of responsive records is sorted according to said record date.

13. The method of claim 7, wherein said list of responsive records is sorted according to said record author.

14. The method of claim 9, wherein said list of responsive records is sorted according to said relevance of said records.

15. The method of claim 6, further comprising the step of resolving ties in a preceding sort.

16. The method of claim 1 , further comprising the step of identifying responsive records that were displayed in their entirety in a prior search.

17. The method of claim 16, wherein said responsive records are identified with text.

18. The method of claim 16, wherein said responsive records are identified with icons.

19. The method of claim 16, wherein said responsive records are identified with color.

20. The method of claim 16, wherein said responsive records are identified with a browser viewed link designation.

21. An apparatus for displaying records responsive to a database query comprising: means for displaying a list of identifiers for a plurality of said responsive records, and

means for displaying selected elements of at least one of said responsive records, wherein said list of identifiers and said selected elements are displayed simultaneously.

22. The apparatus of claim 21, wherein said selected elements comprise the entirety of one of said responsive records.

23. The apparatus of claim 21 , further comprising a means for identifying said responsive records.

24. The apparatus of claim 21, further comprising means for identifying records displayed in their entirety in a prior search.

25. The apparatus of claim 21, further comprising a means for identifying records that were responsive to a prior search.

26. The apparatus of claim 21, further comprising means for sorting said responsive records.

27. The apparatus of claim 26, wherein said means for sorting comprises an algorithm.

28. The apparatus of claim 27, wherein said algorithm comprises:

means for computing the frequency of occurrence of a plurality of search terms; and means for ordering said responsive records based on said frequency.

29. The apparatus of claim 28, wherein said algorithm further comprises: means for computing the proximity of said search terms; and means for ordering said responsive records based on said proximity.

30. The apparatus of claim 27, wherein said algorithm comprises: means for computing the proximity of a plurality of said search terms; and means for ordering said responsive records based on said proximity.

31. The apparatus of claim 27, wherein said algorithm comprises:

means for computing the number of times each of said responsive records is referenced by each other of said responsive records; and

ordering said responsive records based on said number.

32. The apparatus of claim 31, wherein said means for computing comprises a table containing the number of times each of said responsive records is referenced by each other responsive record.

33. The apparatus of claim 27, wherein said algorithm comprises means for ordering said responsive records according to the number of times each of said responsive records has been selected by prior users.

34. The apparatus of claim 21, further comprising means of computing the relevance of said responsive records.

35. The apparatus of claim 21, wherein said list of selected records is sorted according to a record name.

36. The apparatus of claim 21, wherein said list of selected records is sorted according to a record citation.

37. The apparatus of claim 21, wherein said list of selected records is sorted according to a record date.

38. The apparatus of claim 21, wherein said list of selected records is sorted according to a record author.

39. The apparatus of claim 21 , wherein said list of selected records is sorted according to a record rank, said record rank being determined by a previously- applied algorithm.

40. The apparatus of claim 21 , further comprising means for identifying records contained in said list of responsive records that were displayed in their entirety in a prior search.

41. The apparatus of claim 40 wherein said identifying means comprises text.

42. The apparatus of claim 40 wherein said identifying means comprises an icon.

43. The apparatus of claim 40 wherein said identifying means comprises colored text.

44. The apparatus of claim 40 wherein said identifying means comprises a viewed link designation.

45. The apparatus of claim 21, further comprising a means for identifying records contained in said list of responsive records that were selected in a prior search.

46. The apparatus of claim 45 wherein said identifying means comprises text.

47. The apparatus of claim 45 wherein said identifying means comprises an icon.

48. The apparatus of claim 45 wherein said identifying means comprises colored text.

49. The apparatus of claim 45 wherein said identifying means comprises a viewed link designation.

50. A method of sorting a first set of records, comprising the steps of:

for each record in said first set of records, detecting the number of times a component of said record is referenced by records in a second set of records; and

sorting said first set of records based upon said number.

51. The method of claim 50, wherein said first set of records and said second set of records are contained in a database.

52. The method of claim 50. wherein said first set of records and said second set of records are the same.

53. The method of claim 50, wherein said first set of records and said second set of records are comprised of legal documents.

54. The method of claim 50, wherein said component comprises a citation.

55. The method of claim 50, wherein said component comprises a case name.

56. The method of claim 50, wherein said first set of records is sorted in inverse order to said number.

57. The method of claim 50. further comprising the steps of:

dividing said second set of records into classes;

assigning a set of weights, one to each one of said classes; and

for each record in said first set, counting the number of records in each of said classes that reference said record in said first set;

for each class, multiplying said number of records by said weight together to form a composite number for each said class and

adding each of said composite numbers together to compute said number of times said component of said first set record is referenced.

58. The method of claim 50, wherein said first set of records is generated by performing a query on a third set of records.

59. The method of claim 58 wherein the third set of records is comprised of legal documents.

60. The method of claim 58 wherein the third set of records and the second set of records are identical.

61. The method of claim 58, wherein said step of performing a query comprises the step of applying Boolean logic to a keyword.

62. The method of claim 61, wherein a reference to said component is disregarded unless said component is referenced within a predetermined range of said keyword.

63. The method of claim 50, wherein said component is identified by a tag.

64. The method of claim 63, wherein said tag comprises a set of characters defined by Extensible Markup Language.

65. The method of claim 50, further comprising the step of displaying said number.

66. The method of claim 50, further comprising the step of:

creating, prior to said detecting step, a unique identifier for each record in said second set of records, and a first look-up table, wherein

said first look-up table is configured for storing the unique identifiers of all records cited by said record.

67. The method of claim 66, further comprising the step of:

creating, prior to said detecting step, a unique identifier for each record in said second set of records, and a second look-up table, wherein

said second look-up table is configured for storing the unique identifiers of all records citing said record.

68. The method of claim 66, wherein said first look-up table is further configured for storing the position within said record where each other record in said second set of records is cited.

69. The method of claim 67, wherein said second look-up table is further configured for storing the position within said citing records where each said record is cited.

70. The method of claim 67, wherein said second look-up table is further configured to store the number of times said record is cited by each of the other records in said second set of records.

71. The method of claim 67, wherein said first look-up table and said second lookup table are merged into one structure.

72. The method of claim 66, wherein said first set of records and said second set of records are legal documents.

73. The method of claim 67, wherein said first set of records and said second set of records are legal documents.

74. A method of sorting a first set of legal documents, comprising the steps of:

for each legal document in said first set of legal documents, detecting the number of times said legal document is cited by legal documents in a second set of legal documents; and

sorting said first set of legal documents in an order based upon said number.

75. The method of claim 74, wherein said first set of legal documents and said second set of legal documents are the same.

76. The method of claim 74, further comprising the steps of:

dividing said second set of records into classes;

assigning a set of weights, one to each one of said classes; and

for each class, multiplying said number of records by said weight assigned to each said class to form a composite number for each said class; and

77. The method of claim 74, wherein said first set of records is generated by performing a query on said third set of records.

78. The method of claim 77, wherein the third set of records is comprised of legal documents.

79. The method of claim 77, wherein the third set of records and the second set of records are identical.

80. The method of claim 77, wherein said step of performing a query comprises the step of applying Boolean logic to a keyword.

81. The method of claim 80, wherein a reference to said component is disregarded unless said component is referenced within a predetermined range of said keyword.

82. The method of claim 74, wherein said component is identified by a tag.

83. The method of claim 82, wherein said tag comprises a set of characters defined by Extensible Markup Language.

84. The method of claim 74, further comprising the step of displaying said number.

85. A method of identifying additions to a list of records, comprising the steps of:

counting the number of times a record not identified in said list is referenced by the members of said list; and

adding to said list, an identifier for each record for which said number exceeds a predetermined value.

86. The method of claim 85, wherein said records are legal documents.

87. The method of claim 85, wherein said list comprises a list of citations;

88. The method of claim 85, wherein said list of related records is generated by performing a query on a database.

89. The method of claim 88, wherein the step of performing a query comprises applying Boolean logic to a keyword.

90. The method of claim 89, wherein said references are disregarded unless they fall within a predetermined range of said keyword.

91. The method of claim 85, wherein said references are identified b Extensible Markup Language tags.

92. The method of claim 91, wherein said tag comprises a set of characters defined by Extensible Markup Language.

93. The method of claim 85, wherein said counting step is accomplished through the use of a lookup table.

94. The method of claim 85, further comprising the steps of:

dividing said list of records into classes;

assigning a weight to each one of said classes; and

incrementing said number an amount based on said weight for each referenced record.

95. The method of claim 94, further comprising the step of sorting said list of records in an order based upon said number.

96. The method of claim 85, further comprising the step of displaying said number.