AU2014396820B2

AU2014396820B2 - Systems and methods for content on-boarding

Info

Publication number: AU2014396820B2
Application number: AU2014396820A
Authority: AU
Inventors: Hassan Malik; Olof-Ors MANS; Timothy NIXON
Original assignee: Thomson Reuters Enterprise Centre GmbH
Current assignee: Thomson Reuters Enterprise Centre GmbH
Priority date: 2014-06-11
Filing date: 2014-06-11
Publication date: 2017-08-17
Anticipated expiration: 2034-06-11
Also published as: AU2014396820A1; CA2951519C; WO2015191063A1; CA2951519A1

Abstract

The present disclosure is directed towards systems and methods for generating a recommendation to on-board a candidate document to an on-line research system, which comprises receiving from an electronic device, a set of data items associated with a candidate document, the candidate document being a document that is a candidate to be made available via the on-line research system and storing the set of data items in a memory. The systems and methods of the present disclosure then automatically analyze the set of data items using a computer program stored in the memory and generate a recommendation as to whether to obtain or not obtain the candidate document. A signal is then generated and transmitted to the electronic device, the signal based upon the recommendation.

Description

PCT/US2014/041935 WO 2015/191063

SYSTEMS AND METHODS FOR CONTENT ON-BOARDING

COPYRIGHT NOTICE

[0001J A portion of the disclosure of this patent document contains material, which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves ail copyright rights whatsoever. The following notice applies to this document: Copyright © 2014 Thomson Renters.

TECHNICAL FIELD

[0002] This disclosure relates generally to the collection of content. More specifically, the disclosure is directed towards systems and methods for recommending the uploading, also referred to as on-boarding, of candidate documents to an on-line research system.

BACKGROUND

[0003] On-line research systems are invaluable tools that are used in nearly every business, legal, scientific and academic environment For example, in the legal environment, online research systems are continually used by attorneys, court personnel and students in order to continuously research and be kept informed of the most recent legal decisions, statutes and legislation. Indeed, it is not uncommon for a judicial decision issued as recently as the previous day to have a substantial impact on an attorney’s strategy or a court’s legal analysis. Further, it is not uncommon for a judicial decision from a lower court or a different jurisdiction to also have an impact on an attorney’s strategy or a court’s legal analysis.

[0004] Yet, the current methodology employed to upload or on-board documents to an on-line legal research system typically involves a manual process in which an individual, such as a legal runner in a courthouse or a legal sitter monitoring court databases, makes a subjective determination as to whether a candidate document is relevant and should be on-boarded to the on-line Segal research system The risk in using such a methodology is that certain court documents, which may be of great importance to a legal analysis, may never actually be captured and included in the on-line legal research system because the legal runner or sitter did not deem the court document as a relevant document that should be uploaded. Furthermore, by employing this methodology, the candidate court, document may be lost forever, despite a change in 1 PCT/US2014/041935 WO 2015/191063 circutftStaaces or user requirements that may greatly increase the importance and relevancy of the candidate document. These risks are especially prevalent when looking ίο court documents from lower courts or unpopular jurisdictions, where this manual process is employed more often to result in the candidate document not being on-boarded. One theoretical “solution” to this issue Is to simply on-board all documents, including for example, small claim court, decisions, local building code, and the like. Yet such a “solution5' is simply.not .economically feasible given the high potential cost of on-boarding all documents.

[0005] Furthermore, the current methodology employed to on-board documents where court runners are tasked to make the subjective determination as to whether a candidate document is relevant and should be on-boarded to the on-line legal research system creates an issue of inconsistency across the universe of documents that are on-boarded, [0006] Accordingly, there exists a need for automated methods and systems that will make a subjective determination in a quick and efficient manner as to whether a candidate document should be be on-boarded to the on-line legal research system. Further there exists a need for automated methods and systems that will continuously evaluate whether a candidate document should be on-boarded to the on-line legal, research system in view of future events or statistics demonstrating a need for the candidate document.

SUMMARY

[0007] The present disclosure is directed towards systems and methods recommending the on-boarding of candidate documents to an on-line research system. In one aspect, the method includes receiving, from an eleetronic device, a set of data hems associated with a candidate document, the candidate document being a document that is a candidate to be made a vailable via the on-line research system and storing the set of da ta items in a first memory. The set of data items are then automatically analyzed using a computer program stored in the first memory and a recommendation as to whether to obtain or not obtain the candidate document is generated using the computet program, A signal is then generated based upon the recommendation and transmitted to the electronic device.

[0008] According to one embodiment, the method further includes, in response to the step of transmitting the sign al and where in the recommendation is an obtain recommendation, receiving an electronic version of the candi date document from the electroni c device and storinu tbe electronic version in a second memory. PCT/US2014/041935 WO 2015/191063 [0009] In one embodiment* the method further includes, in response to the step of transmitting the recommendation and wherein the recommendation is a do not obtain recommendation, reviewing the set of data items based upon a set of additional information and determining whether to generate a modified recommendation. If a modified recommendation is generated, the method further includes storing the modified recommendation in the first memory, generating a modified signal based upon the modified recommendation and transmitting the modified signal to the electronic device. According to one embodiment, the step of reviewing is ongoing and is triggered by at least one of an event and an end of a time period since the last time the step of reviewing was performed. In one embodiment, the method further includes, in response to the step of transmitting the modified signal and wherein the modified recommendation is an obtain recommendation, receiving an electronic version of the candidate document from the electronic de vice and storing the electronic version in the second memory.

[0010] A system, as well as articles that include a machine-readable medium storing machine-readable program code for implementing the various techniques, are disclosed. Details of various embodiments are discussed in greater detail below.

[0011] Additional features and advantages will be readily apparent from the following detailed description, the accompanying drawings and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] FIG. 1 is a schematic depicting an exemplary computer-based system for generating a recommendation to on-board a candidate document to an on-line research system; [0013] FIG . 2 Is a flow diagram illustrating an exemplary computer-implemented method for generating a recommendation to on-board a candidate document to an on-line research system; [0014] FIG. 3 is a flow diagram illustrating an exemplary computer-implemented method for on-boarding a candidate document to an on-line research system; [0015] FIG. 4 is a flow diagram illustrating an exemplary computer-implemented method for generating a modified recommendation to an on-line research system during a subsequent review; [0016] FIG. 5 is a flow' diagram illustrating a further detailed exemplary computer- implemented method for generating a recommendation to on-board a candidate document to an on-line research system; and PCT/US2014/041935 WO 2015/191063 [0017] FIGS.· 6 and 6A are flow diagrams illustrating a. further detailed exemplary computer-implemented method for generating a modified recommendation to an on-line research system during a subsequent review.

[0018] Like reference symbols in the various drawings indicate like elements,

DET AILED DESCRIPTION

[0019] In the following description, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific embodiments m which the disclosure may be practiced. It is to be understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present disclosure.

[0020] 'fuming now to FIG , 1. an example of a suitable computing system 100 within which embodiments of the disclosure may be implemented is presented. The computing system 100 is only one example and is not intended to suggest any limitation as to the scope of use or functionality of the disclosure. Neither should the computing system 100 be interpreted as having any dependency or requirement relating to any one or combina tion of illustrated components. [0021 ] For example, the present disclosure is operational with numerous other general purpose or special purpose computing consumer electronics, network PCs, minicomputers, mainframe computers, laptop computers, as well as distributed computing environments that include any of the above systems or devices, and the like.

[0022] T he disclosure may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, loop code segments and constructs, and other computer instruction known to those skilled in the art that perform particular tasks or implement particular abstract data types. The disclosure can be practiced in distributed computing environments where tasks are performed by remote processing devices that axe linked through a communications network. !u a distributed computing environment, program modules are located in both local and remote computer storage media including memory storage devices. Tasks performed by the programs and modules are described below and with the aid of figures. Those skilled in the art may implement the description and figures as processor executable instructions, which may be written on any form of a computer readable media. 4 PCT/US2014/041935 WO 2015/191063 [0023] In one embodiment, with reference to FIG. I, the computing system 100 includes a server device 110 configured to include a processor 112, such as a central processing unit (“CPU”), random access memory (“RAM”) 114, one or more input-output devices 116, such as a display device (not shown) and keyboard (not shown), non-volatile memory 120, all of which are interconnected via a common bus 118 and controlled by the processor 112, According to one embodiment, the server 110 is part of an on -line research system. In another embodiment, the server 110 is separate from, the on-line research system and transmits one or more candidate documents to be stored within the on-line research system.

[0024] As shown in the Fig. 1. example, in one embodiment, the non-volatile memory 120 is configured to Include a recommendation module 122, a scoring module 124 and a communication module 126. The scoring module 124 is configured to analyze one or more data items associated with a candidate document on an iterative basis and generate a score for each of the data items using one or more of the rules maintained in a set of predefined scoring patterns 136, as well as a combined overall score for the candidate document using the individual data item scores. The recommendation module 122 is configured to generate a recommendation to obtain or to not obtain a candidate document, using one or more of the rules maintained in the set of predefined scoring patterns 13 6 and the scores generated by the scoring module 124. Lastly, a communication module 126 is provided to receive the set of data items associated with the candidate document, as wel l as any additional information related to tire set of data items, and to generate and transmit, a signal associated with a recommendation to obtain or not obtain the candidate document. Additional details of modules 122, 124 and 126 are discussed in connection with FIGS. 2-5.

[0025] As shown in FIG. 1, in one embodiment, a network 150 is provided that can include various devices such as routers, server, and switching elements connected in an Intranet, Extranet or Internet, configuration. In one embodiment, the network. 150 uses wired communications to transfer information between an access device 160, the server device 110, a data store 130 and content servers 170 and 180. In another embodiment, the network 150 employs wireless communication protocols to transfer information between the access device 160, the server device 110, the data store 130 and the content servers 170 and 180. For example, the network 150 may be a cellular or mobile network employing digital cellular standards including but not limited to the 3GPP, 3GPP2 and AMPS family of standards such as Global 5 PCT/US2014/041935 WO 2015/191063

Systemfor Mobile Communications (GSM), General Packet Radio Service (GPRS). CDMAOne, CDMA2000, Evolirtiou-Data Optimized (EV-DG), LTE Advanced, Enhanced Data Rates for GSM Evolution (EDGE), Universal Mobile Telecommunications System {UMTS), Digital Enhanced Cordless Telecommunications (DECT), Digital AMPS (IS-1 36/TDMA), and integrated Digital Enhanced Network (iDEN). The network 150 may also be a Wide Area Network (WAN), such as the Internet, which employs one or more transmission protocols, e.g. TCP/IP. As another example, the network 150 may employ a combination of digital cellular standards and transmission protocols. In yet other embodiments, the network 150 may employ a combination of wired and wireless technologies to transfer informati on between the access device 160, the server device 110.. the data store 130 and the content servers 170 and 180.

[0026] The data store 1.30 is a repository that maintains and stores information utilized by the before-mentioned modules 122,124 and 126. In one embodiment, the data store 130 is a relational database. In another embodiment, the data store 130 is a directory server, such as a

Lightweight Directory Access .Protocol,(“LDAP”). In yet another embodiment, the data store 130 is an area of non-volatile memory' 120 of the server device 110.

[0027] In one embodiment, as shown in the FIG. 1 example, the data store 130 includes a candidate data item database 132, an on-boarded document database 134 and the set of predefined scoring patterns 136. According to one embodiment, the on-boarded document database 134 maintains the electronic versions of the on-boarded documents, i.e. candidate documents that have been uploaded, or pn-boarded, from, the access device 160. Examples of on-boarded documents include, but are not limited to, court documents, such as judicial decisions, orders and opinions, complaints, answers, briefs, legal memorandum, expert reports, deposition transcripts, trial transcripts, hearing transcripts and party contentions, as well as state and federal statutes, administrative codes, newspaper and magazine articles, public records, law journals, law review's, treatises and legal forms.

[0028] The candidate data item database 132, in one embodiment, maintains the set of data items received by the communication module 126 and used by the scoring module 124 to analyze whether a candidate document should be on-boarded. The set of data items are derived from a candidate document and may include, but are not limited to, the candidate document type, level and jurisdiction of the court, identification of the parties, identification of the court officers, including the judge or judge and counsel for the parties, nature of the legal mater, and subject 6 PCT/US2014/041935 WO 2015/191063 matter data, such as the legal concept at issue, damages data and fact pattern data. In one embodiment, the candidate data item database 132 maintains the set of data items in a structured data store, such as a relational or hierarchal database.

[0029] According to one embodiment, the set of predefined scoring patterns 136 includes one or more scoring rales used by the scoring module 124 to score individual data items and the candidate document itself and by the recommendation module 122 in order to make a determination as to whether to obtain or not obtain a candidate document. In one embodiment, the set of predefined scoring patterns 136 is maintained in a structured data store, such as a relational or 'hierarchal database, and is established by an administrator using the administrator device 190 to determine the scoring patterns to be utilized by the scoring module 124 in scoring individual data items, [0030] Although the data store 130 shown in FIG. I is connected to the network 150, it will be appreciated by one skilled in the art that the data store 130 and/or any of the information shown therein, can be distributed across various servers and be accessible to the server 110 over the network 150; be coupled directly to the server 110; be configured as part of server 110 and interconnected to processor 112. RAM 114, the one or more input-output devices 116 and the non-volatile memory 120 via the common bus 118; or be configured in an area of non-volatile memory 120 of the server 110.

[00311 Content servers 170 and 180 are each configured to include a content server processor, RAM, one or more input-output devices, such as a display device and keyboard, and non-volatile memory, all of which are interconnected via a common bus and controlled by the respective content server processor. According to one embodiment, content servers 170 and 180 provide additional information to the scoring module 124 so that it can perform a subsequent analysis of the set of data items. In one embodiment the content server 170 may be a server that provides news content, such as global or national news, financial news, sporting event news or entertainment news. In another embodiment, the content server .180 may be a library of usage data regarding the ac tivities of users of the on-line research system. In yet another embodiment, the content server 170 and 180 may publicly accessible databases making court documents available, such as New York State Unified Court System’s e-Track website or the United States Supreme Court’s website. 7 PCT/U S2014/041935 WO 2015/191063 [0032] The access device 160, according to one embodiment, is a mobile device having a graphical user interface (“GUI”) 164, a digital signal processor 162, with an application module 162A, internal and external storage components (not shown), a power management system (not shown), an audio component (not shown), audio input/output components (not shown), an image capture and process system (not shown), RF antenna (not shown) and a subscriber identification module (SIM) (not shown). According to another embodiment the access device 160, is a general purpose or special purpose computing device comprising a processor, transient and persistent storage devices, an input/output subsystem, a bus to provide a communications path between components comprising the general purpose or special purpose computer, and a web-based client application, such as a web browser, which allows a user to access the server 110 and the content servers 170 and 180. Examples of web browsers are known in the art, and include well-known web browsers such as such as Microsoft® Internet Explorer®, Google Chrome™, Mozilla Firefox® and Apple® Safari®.

[0033] The administrator device 190, according to one embodiment, is a general purpose or special purpose computing device comprising a processor, transient and persistent storage devices, an input/output subsystem, a bus to provide a communications path between components comprising the general purpose or special putpose computer, and a web-based client application, such as a web browser, which allows a user to access the server 110 and the content servers 170 and 180. Examples of web browsers are known in the art, and include well-known web browsers such as such as Microsoft'® Internet Explorer®, Google Chrome™, Mozilla Fireibx® and Apple®.Safari®. According to another embodiment, administrator device 190 is a mobile device having a Gift (not shown), a digital signal processor with an application module (not shown), internal and external storage components (not shown), a power management system (not shown ), an audio component (not shown), audio input/output components (not shown), an image capture and process system (not. shown), RF antenna (not shown), and a subscriber identification module (SIM) (not shown).

[0034] Further, it should be noted that the system 100 shown in FIG. 1 is only one embodiment of the disclosure. Other system embodiments of the disclosure may include additional structures that are not shown, such as secondary storage and additional computational devices. In addition, various other embodiments of the disclosure include fewer structures than those shown in FIG. 1. For example, in one embodiment, die disclosure is implemented on a 8 PCT/US2014/041935 WO 2015/191063 single computing device in a non-neiworked standalone configuration. Data input and requests are communicated to the computing device via an input device, such as a keyboard and/or mouse. Data output, such as the computed significance score, of the system is communicated from the computing device to a display device, such as a computer monitor, [0035] Turning now to FIG, 2, an exemplary method 200 for on-boarding candidate documents to an on-line research system is disclosed. In the illustrated embodiment shown in FIG. 2, the communication module 126 receives a set of data items associated with a candidate document from the access device 160, step 210. As described previously, examples of a candidate document include, but are not limited, court documents, such as judicial decisions, orders and opinions, complaints, answers, briefs, legal memorandum, expert reports, deposition transcripts, trial transcripts, hearing transcripts and party contentions, as well as state and federal statutes, administrative codes, newspaper and magazine articles, public records, law journals, law reviews, treatises and legal forms. In one embodiment, the access device 160 is a mobile device in which a user sitting in a remote location, such as legal runner in a courthouse, uploads a set of data items associated with the candidate document located at the courthouse via the network 150 to the communication module 126 through the user interface 164. in another embodiment, the access device 160 is a general purpose computer from which a user monitors publicly accessible databases and uploads , via the network 150 to the comm unication mod ule 126, a set of data items associated with a candidate document the user located in the one of the publicly accessible databases.

[0036] In one embodiment, the candidate document is a. court document and the set of data items associated with the candidate document includes the candidate document type, level and jurisdiction of the court, identification of the parties, identification of the court officers, including the judge and or counsel for the parties, and subject matter data, such as the legal concept at issue, damages data and fact pattern data. For example, the candidate document may be a complaint asserting a products liability cause of action filed in the Supreme Court of the State of New York and the set of data items includes: “doc type: first filed complaint,” “jurisdiction: Sale, New York, USA/' “court level: Supreme Court · Commercial Division,” “nature of suit: tort products liability,” “plaintiff's): James Smithson, Anna Smithson” “defendants): ABC Paint Supplies” and “fact pattern data: allegation of ABC Prime X-l paint causing retinal damage to husband, occupation painter” and “damages: compliant seeks $5 M 9 PCT/US2014/041935 WO 2015/191063 US ” It is to. be understood that the type of candidate document, as well as the number and type of data items are not limited to the description disclosed herein and that other candidate document types and data items may be provided.

[0037] According to one embodiment, the user interface 164, in order to receive the set of data items associated with a candidate document, provides a combination of vacant search fields configured to receive text inputs, e.g. a text box, and predefined fields, which include a plurality of predefined input values presented in a drop-down menu configuration. For example, the user interface 164 may provide a predefined field for the “doc type” that includes a drop down menu with a listing of predefined document type values, such as “complaint,” “answer” “answer and coiinter-claim(s)” from which the user may choose, while providing a vacant field for the defendant party data. In another embodiment, the user interface 164 provides only vacant search fields for a user to manually enter tire set of data items associated with the candidate document. In another embodiment, the user interface 164, provides a combination of predefined and vacant input fields, which are a presented to a user in a prioritized sequence, so that the user may input data values in an iterative step process as additional information is needed.

[ 0038] Returning to FIG . 2, in step 212, the set of data items is the stored in memory with in the candidate data item database 132. According to one embodiment, the set of data i tems associated with the candidate document is stored and maintained in a structured document, such as an extensible Markup Language (XML) file. At step 214, according to one embodiment, the set of received data items are then automatically analyzed in order to determine a recommendation to obtain or not obtain the candidate document. In one embodiment, the scoring module 124 analyzes the set of received data items associated with the candidate documen t and determines individual scores for each of the data items in order to subsequently determine an overall score for the candidate document. Details regarding the specific scoring methodology are discussed in connection with FIGS. 5, 6 and 6A. lire scoring module 124 makes its determination using the set of the pre-defined scoring patterns 136. In another embodiment, the shoring module 124 analyzes the set of received data items associated with the candidate document using a logic model, [0039] In step 2 5 6, a recommendation is then generated by the recommendation module 122 to obtain or not obtain the candidate document. According to one embodiment, the 10 PCT/US2014/041935 WO 2015/191063 recommendation -module 122 receives a» overall score for the candidate document and. determines whether the candidate document should be presently obtained or not obtained by the system 100, The recommendation module 122 makes its determination to obtain or not obtain the candidate document based on whether the overall score of the document is greater than a threshold value. In another embodiment, the recommendation module 122 makes its determination based on the overall outcome of a logic model. According to another embodiment, at step 216, the recommendation module 122 first makes a determination, as to whether that is has scores for sufficient number data items to make a recommendation and if not, generates a recommendation requesting additional data items.

[0040] In one embodiment, as depicted in step 218, once the recommendation is generated, the communication module 126 generates a signal associated with the recommendation. Lastly, at step 220, the communication module 1.26 transmits the generated signal to the access device 160. In one embodiment, the communication module 126 transmits the signal immediately upon completion of the generation of the signal, [0041 ] Turning now to FIG. 3, an exemplary compu ter-implemented method 300 for continuing to analyze candidate documents to determine whether they are to on-boarded to an on-line research system is disclosed. In step 310 of the embodiment shown in FIG. 3, the communication module 126 transmits the recommendation generated by the recommendation module 122 to the access device 160. According to one embodiment, the recommendation is one of two possibilities; (i.) 'Obtain Candidate Document” or (it) "Do Not Obtain Candidate Document.” According to another embodiment, the recommendation is one of three possibilities: (i) 'Obtain Candidate Document,” (ii) “Do Not Obtain Candidate Document” or (iii) Supplemental Data items Required,” along with a listing of the requisite supplemental data items, hi the event that supplemental data items are requested, process flow will undergo method 200 and analyze the supplemental data items in conjunction the originally received in order to determine a recommendation to obtain or not obtain the document.

[0042] At step 320, a determination is made as to whether or not to obtain the candidate document. If the recommendation is to obtain the candidate document, process flow continues to step 330, in which an electronic version of the candidate document is generated. According to one embodiment, the user of the access device 160 receives the recommendation and undertakes the process of generating the electronic version of the document. In one embodiment, an 11 PCT/US2014/041935 WO 2015/191063 electronic version the document is generated using techniques well-known in the art such as via use of a conventional paper scanner or a camera on a smartphone or tablet with a corresponding mobile application. Continuing with the previous example, in which the candidate document is a complaint asserting a products liability cause of action filed in the S upreme Court of the State of New York, a legal .runner sitting in the courthouse, having received the recommendation, on his mobile device to obtain the complaint, generates an electronic version of the complaint using a portable scanner connected to his mobile device.

[0043] In step 340, once the electronic version of the candidate document is generated, it is transmitted from the access device 160 to the communication module 126, step 340. Continuing with the previous example, tire scanned electronic version of the complaint is uploaded overtire network 150 from tire legal runner's mobile device to the communication module 126. In step 350, the electronic version of the candidate document is then stored in memory within candidate document database 134, [0044] Alternatively, at step 320, if the recommendation is to not obtain the candidate document, then the set of data items associated with the candidate document is maintained in the candidate data item database 132 for subsequent, review as shown in step 360. At step 370, the set of data items associ ated with the candidate document are later analyzed in view of additional information related to the set of data items. According to one embodiment, additional information related to the set of data item includes an increase in notoriety for a given data item. For example, where one of the data items for a candidate court document is the identity of a defendant corporation, additional information discovered during a subsequent review "Would include the fact that the defendant corporation has filed, documents with the Securities and Exchange Commission evidencing their intention to go public. Additional information related to the set of data items, in one embodiment, is obtained, from content servers 170 and 180. For example content server 1.70 may be a financial news wire that provides headlines regarding financial events globally, [0045] in one embodiment, the scoring module 124 analyzes the set of received data items 'associated with the candidate document in view of the related additional information and determines a modification, e.g. an increase or decrease, to each of the individual scores for each data item in order to subsequently determine an overall modified score for the candidate document. Details regarding the specific scoring methodology are discussed in connection with PCT/US2014/041935 WO 2015/191063 FIGS. 6 and 6A. The scoring module 124 makes its modified determination «sing the set of the pre-defined scoring patterns 136. In another embodiment the scoring module 124 analyzes the set of received data items in view of the additional information using a logic model. Continuing with the previous example, the fact that the defendant corporation has filed documents with the Securities and Exchange Commission evidencing their intention to go public would alter and increase the score for the data item, “party identity,” which in turn alters and increases the overall score of the candidate court document. This is because there is typically a higher level of interest in public companies as opposed to privately held entities.

[0046] Returning to FIG. 3, at step 380, a determination is then made as to whether to generate a modified recommendation to obtain the candidate document. According to one embodiment, the recommendation module 122 receives a modified overall score for the candidate document and deiemtin.es at this juncture whether the candidate document should be presently obtained or not obtained bv the system 100. The recommendation module 122 makes its modified determination to obtain or not obtain the candidate document based on whether the overall score of the document is greater than a threshold value. Continuing with the previous example, the increase in the score for the data item, “party identity” in turn increased the overall score of the candidate court document, making the overall score greater than a threshold value and changing the recommendation from a “do not obtain candidate document” to an “obtain candidate document.” [0047] If a modified recommendation is generated, then process flow returns to. step 330, in which method 300 repeats the process of generating and transmitting an electronic version of the candidate document based on the modified recommendation. Conversely, if a determination is made to retrain from generating a modified recommendation to obtain the candidate document, process flow continues to step 360, in which the set of data items associated with the candidate document continue to be maintained in the candidate data Item database 132 for subsequent review.

[0048] Turning now to FIG. 4, an exemplary computer-implemented method 400 for generating a modified recommendation to an on-line research system dining a subsequent review is disclosed. In step 410 of the embodiment shown in FIG. 4, the set of data items associated with the candidate document is maintained in the candidate data item database 132 for subsequent review, step 410. 13 PCT/US2014/041935 WO 2015/191063 [0049] Thesubsequent review is initiated by a triggeringoccurrence, such as the occurrence of an. event, or a scheduled periodic review. According to one embodiment, a scheduled periodic review is a subsequent review to occur on a periodic basis, such as annually, monthly, weekly, or daily. The defined periodic review rules, such as the frequency of the subsequent review, are maintained in the set of pre-defined scoring patterns 136. in one embodiment, an administrator, using the administrator device 190, defines the rules for the periodic review and has the ability to modify the periodic review rules to alter the frequency of the subsequent review as desired.

[0050] According to one embodiment, an event, as disclosed herein, is a significant occurrence relating to one or more of the data items associated with the candidate document, such as a national news event, financial news event, sporting news event, legal news event and/or a legal concept event. For example, where the data item for a candidate document is the identity of the plaintiff husband and wife, and an event would be a widespread news report that the plaintiff husband has been romantically involved with a famous Hollywood actress.

[0051 ] In one embodiment, the occurrence of an event is defined by a set of rules maintained within the set of pre-defined scoring patterns 136. A rule maybe to obtain the candidate document if the number of times a data item appears in a news story is higher than a threshold value. For example, if a rule is that a judge’s identity appears in more than five thousand news articles, where a judge is nominated to the United States Supreme Court and hence here name appears in more approximately twelve thousand news articles, the nomination would be defined as an event since the data item, in this example the judge’s name, appeared in more than five thousand news articles.

[0052] Another rule might be to obtain the candidate document if the data item is related to a change in a legal concept. In another example, where the data item is the nature of the suit, in this example fair labor standards act, a ruling by the United States Supreme Court regarding the definition of “principals activities” under the Fair Labor Standards Act is defined as an event as it involves a major ruling regarding the Fair Labor Standards Act by the Supreme Court. In another example, the occurrence of an event may constitute a rule Set up. by an administrator at the administrator device 190 based on usage data from an on-line research system, such as the data usage demonstrating that in the last year, users of the on-line research system have searched for three times more bankruptcy cases than in the previous two years. PCT/US2014/041935 WO 2015/191063 [0053] In one embodiment* information regarding events is received front content servers 170 and 180. For example, content server 170 may be a newswire that provides a variety of headlines, such as headlines related to scandals in the entertainment industry or the website of the United States Supreme Court posting their most recent decisions. Content server 180 may, for example* comprise a library of usage data regarding the activities of users with an on-line research system.

[0054] Returning to FIG, 4, in step 412, a determination is made as to whether an event occurred. In one embodiment, the scoring module 124 identifies the occurrence of an event based on the rules maintained In the set of pre-defined scoring patterns 136. For example, the scoring module 124 determines an event occurrence of a party’s name appearing in the news more than a threshold value from the content server 170, which maintains such information from the newswire feed. If an event has not occurred, then process flow returns to step 410 and the set of data items continue to be maintained in the candidate data item database 132. Alternati vely, in step 414* a determination is made as to whether a periodic review is scheduled. For example, scoring module 124 determines whether it been three months since the last periodic review or since the “do not obtain” recommendation was first transmitted based on information maintained in the set of pre-detined scoring patterns 136. If the scheduled time has not occurred, then process flow returns to step 410 and the set of data items continue to be maintained in the candidate data item database 132.

[0055] If, however, an event or the scheduled time for the periodic review has occurred, then process flow continues to step 416, in which additional information related to the set of data items associated with the candidate document is received. As disclosed previously, according to one embodimen t, additional information related to the set of data item includes an increase in notoriety for a given data item . For example, the fact that a defendant corporation has filed documents with the Securities and Exchange Commission evidencing their intention to go public. At step 418, the set of data items associated with the candidate document is then analyzed in view of the additional information related to the set of data items. As disclosed previously, in one embodiment, the scoring module 124 having analyzed the set of received data items in view of the related .additional information, makes a determination as to whether the individual scores for each data item should be increased or decreased, in order to subsequently determine an overall modified score for the candidate document. PCT/US2014/041935 WO 2015/191063 [0056] In step 4:20, a determination is then made as whether to generate a modified recommendation to obtain the candidate document based upon the additional information. According to one embodiment, the recommendation module 122 receives a modified overall score for the candidate document and determines at this juncture whether the candidate document should be presently obtained or not obtained by the system 100, The recommendation module 122 makes its modified determination to obtain or not obtain the candidate document based on whether the overall score of the document is greater than a threshold value, if a «Μ-· .... determination is made to refrain from generating a modified “obtain5’ recommendation, the process flow returns to step 410, in which the set of data items associated with the candidate document is then maintained in the candidate data item database 132 for subsequent review.

[0057] Otherwise, if a modified recommendation is generated to obtain the candidate document, then process flow continues to step 422 and the modified recommendation is transmitted to the access device 160, Subsequently, a t step 424, an electronic version of the candidate document is generated and transmitted from the access device 160 to the communication module 126. At step 426, the electronic version of the candidate document is stored in memory within the candidate document database 134.

[0058] fuming now to FIG. 5, a further detailed exemplary computer-implemented method 500 for generating a recommendation to on-board a candidate document to an on-line research system is disclosed. In the illustrated embodiment shown in FIG. 5, the scoring module 124 analyzes the set of data items associated with a candidate document from the access device 160, step 510. In steps 512 though 519, a series of scores is then determined for the candidate document from the set of associated data items. According to one embodiment, a case-type score, a damages score, a party score, a participant score and a uniqueness score are determined for the candidate document by the scoring module 124, via steps 512-519, based on a plurality of defined rules maintained in the set of pre-deftned patterns 136.

[0059] A case-type score is a score assigned to the candidate document based on the data item for the nature of the suit within the court document. For example, the set of pre-deftned patterns 136 may include a set of defined rules that sets forth that all intellectual property suits, professional negligence suits, class action suits, medical malpractice suits, and products liability suits are to be assigned a ease-type score of 3; all fraud, breach of contract, bankruptcy and 16 PCT/US2014/041935 WO 2015/191063 employment discrimination suits ate to receive a score of 2: and all divorce, premises liability and motor vehicle suits are to receive a score of 1.

[0060] A party score is a score assigned to the candidate document based on the data item for the identification of the party in the court document. For exampl e, the set of pre-defined patterns 136 may include a set of defined rules that dictates that any party that is a Fortune 100 company or a specific government agency, such as the Securities and Exchange Commission or the United States Attorney General’s office, receives a score of 3; any state government. Fortune 500 company, publicly traded company or individual with celebrity status receives a score of 2; and any private individual or small privately held company receives a score of 1.

[0061 ] A participant score is a score assigned to the candidate document based on the data item for the identification of the any participants in the court document, such as the judge, counsel, a party’s counsel, or a party’s counsel’s law firm. For example, the set of pre-defmed patterns 136 may include a set of defined rules that mandates that any large law firm, e,g. more than 500 attorneys, receives a score of 3; any medium-sized law firm receives a score of 2; and any small firm or solo practitioner receives a score of 1. A damages score is a score assigned to the candidate document based on the damages awarded or sought in the court document. For example, the set of pre-defined patterns 136 may include a set of defined rules that cases wherein damages sought or awarded are in excess of US $10 Million are to receive a score of 3 , cases wherein damages awarded or sought are between US$1 Million and US $ 10 Million are to receive a score of 2, any cases wherein any damages awarded or sought are less than US SI Million are to receive a score of 1.

[0062] A uniqueness score is a score assigned to the candidate document based on the data items associated with the fact pattern of the candidate document. For example, a uniqueness score may be based on the products at issue, the legal concept at issue or the geographic location at issue. The set of pre-defined patterns 136 may include a set of defined rules that sets forth that varying numerical score values from 1 to 3 for these individual characteristics. For example, if the candidate document involves a pharmaceutical drug, the uniqueness score is scored higher than a court document pertaining to a malfunctioning^ appliance. Similarly, if the candidate document relates to a hotly contested legal concept, such as a criminal case involving the accidental discharge of a firearm by a minor, which touches on gun control, the uniqueness score 17 PCT/US2014/041935 WO 2015/191063 is scored higher than a court document pertaining to an assault and battery incident having taken place at a night dub without the use of weapons.

[0063] it is to be understood that the number and types of scores determined for a candidate document are not limited to the number and types of scores described herein, which are being disclosed solely to serve as exemplary score types, and that other score types may be determi ned from the set of data items associated with the candidate document by the scoring module 124. Further, the examples of each of the score types disclosed herein are presented purely for illustrative purposes and are not intended to limit the exemplary score disclosed herein. Additionally, it is important to note that the scoring scale is not limited to a 1 to 3 numerical scale, hut can utilize any variation of a scoring scale, as well any other methods known in the art for scoring, [0064 ] Returning to FIG . 5, the scores are then aggregated at step 520 by the scoring module 124. In one embodiment, the individual scores are aggrega ted by adding each of the individual scores. For example, the case-type score, the damages score, the party score, the participant score and the uniqueness score would be added with the total sum being the overall score for the candidate complaint document. In another embodiment, the individual scores are aggregated using a weighted sum model. For example, using a weighted sum model, the case-type score, the damages score and the party score would be weighted higher than the participant score and the uniqueness score in determining the overall score of the candidate document.

[0065] Next, at step 530, a determination is made by the recommendation module 122 as to whether the combined score is greater than a threshold value. In one embodiment, the set of pre-defmed patterns 136 includes a defined .rule that sets forth the overall threshold numerical score that is to be used by the recommendation module 122 to generate an “obtain” or a “do not obtain” recommendation. For example, where the candidate document is a complaint asserting a products liability cause of action and the set of data items includes "‘nature of suit: tort products liability” being assigned a case-type score of 3; ‘‘plaintiff!s); lames Smithson, Anna Smithson” and “defendants): ABC Paint Supplies” being assigned a party score of 1; “damages: compliant seeks $5 M US” being assigned a damages score of 2 and “fact pattern data: allegation of ABC Prime X-1 paint causing retinal damage to husband, occupation painter5’ being assigned a uniqueness score of 1, an overall, score would be the sunt total of the individual scores, in this example, an overall score of 7. Further, the threshold value to obtain a document may be defined PCT/US2014/041935 WO 2015/191063 within the set of pre-defined patterns i 36 may be 8, in which case the recommendation module 122 would determine that, the candidate document should not be obtained [0066] If the combined score is greater than the threshold value, an “obtain” recommendation is generated at step 540 and process flow then continues to step 542, in which a signal associated with the “obtain” recommendation is generated and transmitted by the communication module 126 to the access device 160. Subsequently, at step 544, an electronic version of the candidate document is generated and received .from, the access device 160 and stored in the on-boarded document database 134 and the process flow ends, it should be noted that once an “obtain” recommendation is generated and the candidate document is ultimately obtained and stored, the set of data items associated with the candidate document also continues to be maintained in the candidate item database 132 for subsequent analysis.

[0067] Returning to FIG. 5, if the combined score is less than the threshold value, a “Do Not Obtain55 recommendation is generated at step 550 and process flow then continues to step 552, in which the set of data items associated with the candidate document continue to be maintained in candidate item database 132.

[0068] Taming now to FIGS. 6 and 6A, a further detailed exemplary computer-implemented method for generating a modified recommendation to an on-line research system during a subsequent review is disclosed . In step 610 of the embodiment shown in FIG. 6, the scoring module 124 analyzes the set of data hems associated with the candidate document in view of the additional information recei ved from the content servers 170. and 180. For example, content server 170 may be a news wire that provides a variety of headlines, such as headlines related to scandals in the entertainment industry or the website of the United States Supreme Court posting their most recent decisions. Content server 180 may, for example, a library of usage data regarding the activities of users with an on-line research system.

[0069] Individual data items from the set. of data items associated with the candidate document are then analyzed based upon the received additional information. According to one embodiment, as shown via decision boxes 612-619, the individual data items analyzed include the party data, -court officer data, product data, fact pattern data and legal concept data in order to determine whether there has been any change to the set of data items.

[0070] In one embodiment, at step 612, in the instance where the candidate document is a judicial order, a determination is made as to whether any of the parties in the candidate document 19 PCT/US2014/041935 WO 2015/191063 has gained notoriety. An example of this would be when the plaintiff or defendant to the legal proceeding in the candidate document has gained popularity in the news. For example, in a divorce proceeding between plaintiff w ife and defendant ex-husband, i t is determined that the plaintiff wile has gained notoriety because she had later become hugely popular as an A-list Hollywood actress, if a determination is made that the party has gained notoriety'; the party score is increased at step 620. Continuing with the previous example, the party score for the plaintiff wife in the divorce decree increased front a score of 1 to 3 because she has become a popular Hollywood actress often mentioned in the news.

[0071] S imilarly, at step 614, a determination i s made as to whether any of the participants from the candidate document has gained notoriety. T his may occur when the judge, counsel, counsel’s firms, experts or relevant third-parties have gained notoriety. Examples of participants gaining notoriety include where a judge is elevated to a higher court or decides to abandon his judicial career to pursue a political career, or a pharmaceutical manufacturer involved in a products liability action files documents with the Securities and Exchange Commission to become a publ icly traded company. If a determination is made that the participant has gained notoriety, the participant score is increased at step 622. | At step 616, a determination is made as to whether the product at issue from the candidate document has gained notoriety. One example of a product gaining notoriety is where the candidate document pertains to a products liability case involving a defective tire, and later a series of class action suits are filed involving the defective tire. If a determination is made that the product at issue has gained notoriety , the uniqueness score is increased at step 624. Similarly, at step 618, a determination is made as to whether the fact pattern from the candidate document has gained notoriety. On example of a fact pattern gaining notoriety is in a civil suit charged against a military· organization for its use of interrogation tactics and the military’s use of interrogation techniques are later displayed in a documentary that receives world-wide attention stirring a global debate. If a determination is made that the fact pattern has gained notoriety, the uniqueness score is increased at step 626, [0073] At step 619, a determination is made as to whether there was change in legal concept discussed in the candidate document. For example, the candidate documents pertain to a court’s decision on divided patent infringement and later the United States Supreme Court reverses the country’s long standing doctrine on what is required to prove divided infringement. PCT/US2014/041935 WO 2015/191063

If a determination is made that the legal concept has changed, the case-type score is increased at step 628.

[0074] Turning now to FIG. 6 A, if any of the scores have been modified, eg. increased because of a change in notoriety for the individual data items, the scoring module 124 aggregates the individual scores, including any increased scores at step 640, As disclosed previously, the scores are aggregated by summing the individual scores according to one embodiment. In another embodiment, the scores are aggregated using a weighted sum mode. Next, at. step 650, a determination is made by the recommendation module 122 as to whether the combined score is greater than the threshold value. In one embodiment, the set of pre-deb ned patterns 136 includes a defined rule that sets forth the overall threshold numerical score that continues to be used by the recommendation module 122 to generate a modified “obtain” or a “do not obtain” recommendation. For example, where the candidate document is a complaint asserting a products liability cause of action for defective paint and the set of data items includes the identity of the plaintiff who has since been charged as a serial murder and is claiming an insanity, the party score would be increased because of the increase in notoriety of the plaintiff, and the uniqueness score would increase as well because its alleged in his defense that the insanity was caused by the use of the defective paint, having thereby increased the notoriety of the product at issue in the candidate document. Continuing in this example, the uniqueness score and party score would each be increased to a score of 3, modifying the overall score of the candidate document to an 11, which would be then compared to threshold value of 8 maintained in the set of pre-deflned scoring patterns 136.

[0075] If the combined score is greater than the threshold value, an “obtain” recommendation is generated at step 660 and process flow then continues to step 662, in which a signal associated with the “obtain” recommendation is generated and transmitted by the communication module 126 to the access device 160. Subsequently, at step 664, an electronic version of the candidate document is generated and received from the access device 160 and stored in the on-boarded document database 134, Alternati vely, if the combined score is less than the threshold value, a “do not obtain” recommendation is generated at step 670 and process flow then continues to step 672, in which the set of data items associated with the candidate document continue to he maintained in candidate item database 132, PCT/US2014/041935 WO 2015/191063 [0076] Returning how'to FIG. 6, if there has not been any change to the set of data i tems, e.g. none of the patties, participants, products, fact patterns or legal concepts have changed, then process flow continues to step 630, in which a “do not obtain” recommendation is generated and then the set of data items associated with the candidate document continue to be main tained in candidate item database 132 via step 632.

[0077] It is to be understood that the number and types of data items analyzed and scores determined for a candidate document in this disclosed subsequent review are not limited to the number and types of scores described herein, which are being disclosed herein as exemplary, and that, other data items may be analyzed and other score types may be determined from the set of data items associated with the candidate document by the scoring module 124.

[0078] FIGS. 1 through 6A are conceptual illustrations allowing for an explanation of the presen t disclosure, it should be understood that various aspects of the embodiments of the present disclosure could be implemented in hardware, firmware, software, or combinations thereof, hi such embodiments, the various components and/or steps would be implemented in hardware, firmware, and/or software to perform the functions of the present disclosure. That is, the same piece of hardware, firmware, or module of software could perform one or more of the illustrated blocks (e.g., components or steps), [0079] In software implementations, computer software (e.g., programs or other instructions) and/or data is stored on a machine readable medium as part of a computer program product, and is loaded into a computer system .or other device or machine via a removable storage drive, hard drive, or communications interface. Computer programs (also called computer control logic or computer readable program code) are stored in a main and/or secondary memory, and executed by one or more processors (controllers, or the like) to cause the one or more processors to perform the functions of the disclosure as described herein, in this document, the terms “machine readable medium,” “computer program medium” and “computer usable medium” are used to generally refer to media such as a random access memory (RAM); a read only memory' (ROM); a removable storage unit (e.g., a magnetic or optical disc, flash memory device, or the like); a bard disk; or the like.

[0080] Notably, the figures and examples above are not meant to limit the scope of the present disclosure to a single embodiment, as other embodiments are possible by way of interchange of some or all of the described or illustrated elements. Moreover, where certain PCT/US2014/041935 WO 2015/191063 elements of the present disclosure can be partially or fully i mplemented using known components, only those portions of such known components that are necessary for an understanding of the present disclosure are described, and detailed descriptions of other portions of such known components are omitted so as not to obscure the disclosure. In the present specification, an embodiment showing a singular component should not necessarily be limited to other embodiments including a plurality of the same component, and vice-versa, unless explicitly stated otherwise herein. Moreover, applicants do not intend for any term in the specification or claims to be ascribed an uncommon or special meaning unless explicitly set forth as such. Further, the present disclosure encompasses present and future known equivalents to the known components referred to herein by wav of illustration.

[GOBI] The foregoing description of the specific embodiments so fully reveals the general nature of the disclosure that others can, by applying knowledge within the skill of the relevant art(s), readily modify and/or adapt for various applications such specific embodiments, without undue experimentation, without departing from the general concept of the present disclosure. Such adaptations and modifications are therefore intended to be within the meaning and range of equi valents of the disclosed embodiments, based on the teaching and guidance presented herein.

It is to be understood that the phraseology or terminology herein is for the purpose of description and not of limitation, such tha t the terminology or phraseology of the present specification is to be interpreted by the skilled artisan in light of the teachings and guidance presented herein, in combina tion with the knowledge of one skilled in the relevant art(s).

[0082] While various embodiments of the present disc l osur e have been described abo ve, it should be understood that they have been presented by way of example, and not as limitations. It would be apparent to one skilled in the relevant art(s) that various changes in form and detail could be made therein without departing from the spirit and scope of the disclosure. Thus, the present di sclosure should not 'be l imited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented method for recommending the on-boarding of candidate documents to an on-line research system, the computer-implemented method comprising: receiving, from an electronic device, a set of data items associated with a candidate document, the candidate document being a document that is a candidate to be made available via the on-line research system; storing the set of data items in a first memory; automatically analysing the set of data items using a computer program stored in the first memory; generating, using the computer program, a recommendation as to whether to obtain or not obtain the candidate document; generating a signal based upon the recommendation; transmitting the signal to the electronic device; in response to the step of transmitting the signal and wherein the recommendation is an obtain recommendation, receiving an electronic version of the candidate document from the electronic device; and storing the electronic version in a second memory.
2. The computer-implemented method of claim 1, wherein automatically analysing the set of data items using a computer program stored in the first memory further comprises: determining a score for each of one or more data items from the set of data items according to one or more pre-defined scoring patterns; and aggregating the one or more scores for the one or more data items in order to calculate an overall score for the candidate document.
3. The computer-implemented method of claim 2, wherein generating, using the computer program, a recommendation as to whether to obtain or not obtain the candidate document further comprises: determining that the overall score for the candidate document is equal to or greater than a threshold value, the threshold value being defined by the one or more pre-defined scoring patterns; and generating a recommendation to obtain the candidate document.
4. The computer-implemented method of claim 2, wherein generating, using the computer program, a recommendation as to whether to obtain or not obtain the candidate document further comprises: determining that the overall score for the candidate document is less than a threshold value, the threshold value being defined by the one or more pre-defmed scoring patterns; and generating a recommendation to not obtain the candidate document.
5. The computer-implemented method of any one of the preceding claims further comprising: in response to the step of transmitting the recommendation and wherein the recommendation is a do not obtain recommendation, reviewing the set of data items based upon a set of additional information; and based upon the step of reviewing, determining whether to generate modified recommendation and, if a modified recommendation is generated: storing the modified recommendation in the first memory; generating a modified signal based upon the modified recommendation; and transmitting the modified signal to the electronic device.
6. The computer-implemented method of claim 5, wherein the step of reviewing is ongoing and is triggered by at least one of: an event; and an end of a time period since the last time the step of reviewing was performed.
7. The computer-implemented method of claim 6, wherein the time period is selected from a day, a week, a month, and a year.
8. The computer-implemented method of claim 6 or 7, wherein the event comprises one or more of a national news event, financial news event, sporting news event, legal news event and a legal concept event.
9. The method of any one of claims claim 5 to 8 further comprising: in response to the step of transmitting the modified signal and wherein the modified recommendation is an obtain recommendation, receiving an electronic version of the candidate document from the electronic device; and storing the electronic version in the second memory.
10. The computer-implemented method of any one of claims 5 to 9, wherein reviewing the set of data items based upon a set of additional information further comprises: determining a modified score for each of one or more data items from the set of data items according to one or more pre-defmed scoring patterns based upon the set of additional information; and aggregating the one or more modified scores for the one or more data items in order to calculate a modified overall score for the candidate document.
11. The computer-implemented method of any one of claims 5 to 10, wherein determining whether to generate a modified recommendation further comprises: determining that the modified overall score for the candidate document is equal to or greater than a modified threshold value, the modified threshold value being defined by the one or more pre-defmed scoring patterns; and generating a modified recommendation to obtain the candidate document.
12. The computer-implemented method of any one of claims 5 to 11, wherein determining whether to generate modified recommendation further comprises: determining that the modified overall score for the candidate document is less than a modified threshold value, the modified threshold value being defined by the one or more pre-defmed scoring patterns; and generating a modified recommendation to not obtain the candidate document.
13. The computer-implemented method of any one of claims 5 to 12, wherein the set of additional information comprises information identifying any increase in notoriety to a given data item from the set of data items.
14. The computer-implemented method of any one of the preceding claims, wherein the electronic device is a mobile device.
15. Non-transitory computer readable media comprising program code stored thereon for execution by a programmable processor to perform a method for recommending the on-boarding of candidate documents to an on-line research system, the computer readable media comprising: program code for receiving, from an electronic device, a set of data items associated with a candidate document, the candidate document being a document that is a candidate to be made available via the on-line research system; program code for storing the set of data items in a first memory; program code for automatically analysing the set of data items using a computer program stored in the first memory; program code for generating a recommendation as to whether to obtain or not obtain the candidate document; and program code for generating a signal based upon the recommendation; and program code for transmitting the signal to the electronic device. in response to the execution of the program code for transmitting the signal and wherein the recommendation is an obtain recommendation, program code for receiving an electronic version of the candidate document from the electronic device; and program code for storing the electronic version in a second memory.
16. The computer readable media of claim 15, wherein program code for automatically analysing the set of data items using a computer program stored in the first memory further comprises: program code for determining a score for each of one or more data items from the set of data items according to one or more pre-defined scoring patterns; and program code for aggregating the one or more scores for the one or more data items in order to calculate an overall score for the candidate document.
17. The computer readable media of claim 16, wherein the program code for generating a recommendation as to whether to obtain or not obtain the candidate document further comprises: program code determining that the overall score for the candidate document is equal to or greater than a threshold value, the threshold value being defined by the one or more predefined scoring patterns; and program code generating a recommendation to obtain the candidate document.
18. The computer readable media of claim 16, wherein the program code for generating a recommendation as to whether to obtain or not obtain the candidate document further comprises: program code for determining that the overall score for the candidate document is less than a threshold value, the threshold value being defined by the one or more pre-defined scoring patterns; and program code for generating a recommendation to not obtain the candidate document.
19. The computer readable media of any one of claims 15 to 18 further comprising: in response to the execution of the program code for transmitting the recommendation and wherein the recommendation is a do not obtain recommendation, program code for reviewing the set of data items based upon a set of additional information; and based upon the execution of the program code for reviewing, program code for determining whether to generate a modified recommendation and, if a modified recommendation is generated: program code storing the modified recommendation in the memory; program code generating a modified signal based upon the modified recommendation; and program code transmitting the modified signal to the electronic device.
20. The computer readable media of claim 19, wherein the execution of program code for reviewing is ongoing and is triggered by at least one of: an event; and an end of a time period since the last time the step of reviewing was performed.
21. The computer readable media of claim 20, wherein the time period is selected from a day, a week, a month, and a year.
22. The computer readable media of claim 20 or 21, wherein the event comprises one or more of a national news event, financial news event, sporting news event, legal news event and a legal concept event.
23. The computer readable media of any one of claims 19 to 22 further comprising: in response to the execution of program code for transmitting the modified signal and wherein the modified recommendation is an obtain recommendation, program code for receiving an electronic version of the candidate document from the electronic device; and program code for storing the electronic version in the second memory.
24. The computer readable media of any one of claims 19 to 23, wherein program code for reviewing the set of data items based upon a set of additional information further comprises: program code for determining a modified score for each of one or more data items from the set of data items according to one or more pre-defined scoring patterns based upon the set of additional information; and program code for aggregating the one or more modified scores for the one or more data items in order to calculate a modified overall score for the candidate document.
25. The computer readable media of claim 24, wherein program code for determining whether to generate a modified recommendation further comprises: program code for determining that the modified overall score for the candidate document is equal to or greater than a modified threshold value, the modified threshold value being defined by one or more pre-defined scoring patterns; and program code for generating a modified recommendation to obtain the candidate document.
26. The computer readable media of claim 24, wherein program code for determining whether to generate a modified recommendation further comprises: program code for determining that the modified overall score for the candidate document is less than a modified threshold value, the modified threshold value being defined by the one or more pre-defined scoring patterns; and program code for generating a modified recommendation to not obtain the candidate document.
27. The computer readable media of any one of claims 19 to 26, wherein the set of additional information comprises information identifying any increase in notoriety to a given data item from the set of data items.
28. The computer readable media of any one of claims 15 to 27, wherein the electronic device is a mobile device.
29. A system for recommending the on-boarding of candidate documents to an online research system, the system comprising: a data repository comprising a first memory and a second memory; and a server including a processor configured to: receive, from an electronic device, a set of data items associated with a candidate document, the candidate document being a document that is a candidate to be made available via the on-line research system; store the set of data items in the first memory; automatically analyse the set of data items using a computer program stored in the first memory; generate, using the computer program, a recommendation as to whether to obtain or not obtain the candidate document; and generate signal based upon the recommendation; and transmit the signal to the electronic device. in response to the step of transmitting the signal and wherein the recommendation is an obtain recommendation, receive an electronic version of the candidate document from the electronic device; and store the electronic version in a second memory.
30. The system of claim 29, wherein the server, in automatically analysing the set of data items using a computer program stored in the first memory, is further configured to: determine a score for each of one or more data items from the set of data items according to one or more pre-defined scoring patterns; and aggregate the one or more scores for the one or more data items in order to calculate an overall score for the candidate document.
31. The system of claim 30, wherein the server, in generating a recommendation as to whether to obtain or not obtain the candidate document, is further configured to: determine that the overall score for the candidate document is equal to or greater than a threshold value, the threshold value being defined by the one or more pre-defined scoring patterns; and generate a recommendation to obtain the candidate document.
32. The system of claim 30, wherein the server, in generating a recommendation as to whether to obtain or not obtain the candidate document, is further configured to: determine that the overall score for the candidate document is less than a threshold value, the threshold value being defined by the one or more pre-defined scoring patterns; and generate a recommendation to not obtain the candidate document.
33. The system of any one of claims 29 to 32, wherein the server is further configured to: in response to the step of transmitting the recommendation and wherein the recommendation is a do not obtain recommendation, review the set of data items based upon a set of additional information; and based upon the step of reviewing, determine whether to generate a modified recommendation and, if a modified recommendation is generated: store the modified recommendation in the first memory; generate a modified signal based upon the modified recommendation; and transmit the modified signal to the electronic device.
34. The system of claim 33, wherein the reviewing step performed by the server is ongoing and is triggered by at least one of: an event; and an end of a time period since the last time the step of reviewing was performed.
35. The system of claim 34, wherein the time period is selected from a day, a week, a month, and a year.
36. The system of claim 34 or 35, wherein the event comprises one or more of a national news event, financial news event, sporting news event, legal news event and a legal concept event.
37. The system of any one of claims 33 to 36, wherein the server is further configured to: in response to the step of transmitting the modified signal and wherein the modified recommendation is an obtain recommendation, receive an electronic version of the candidate document from the electronic device; and store the electronic version in the second memory.
38. The system of any one of claims 33 to 37, wherein the server in reviewing the set of data items based upon a set of additional information, is further configured to: determine a modified score for each of one or more data items from the set of data items according to one or more pre-defmed scoring patterns based upon the set of additional information; and aggregate the one or more modified scores for the one or more data items in order to calculate modified overall score for the candidate document.
39. The system of claim 38, wherein the server in determining whether to generate a modified recommendation, is further configured to: determine that the modified overall score for the candidate document is equal to or greater than a modified threshold value, the modified threshold value being defined by the one or more pre-defined scoring patterns; and generate a modified recommendation to obtain a candidate document.
40. The system of claim 38, wherein the server in determining whether to generate a modified recommendation, is further configured to: determine that the modified overall score for the candidate document is less than a modified threshold value, the modified threshold value being defined by the one or more predefined scoring patterns; and generate a modified recommendation to not obtain the candidate document.
41. The system of any one of claims 33 to 40, wherein the set of additional information comprises information identifying any increase in notoriety to a given data item from the set of data items.
42. The system of any one of claims 29 to 41, wherein the electronic device is a mobile device.