WO2007142941A2

WO2007142941A2 - System and method for providing network source information

Info

Publication number: WO2007142941A2
Application number: PCT/US2007/012694
Authority: WO
Inventors: Eric Malawer; Benjamin T. Kasoff
Original assignee: Deepmile Networks, Llc
Priority date: 2006-05-30
Filing date: 2007-05-30
Publication date: 2007-12-13
Also published as: WO2007142941A3

Abstract

A system and method for providing network source information manages content by managing sources of content to establish source validation and reliability. An end-to-end information management platform interfaces with structured and unstructured content through public and proprietary networks. The platform identifies collects, processes, analyzes, produces, and disseminates content through a validated network of sources. Valid sources are developed and built-out to establish a collaborative human source information network. User feedback regarding the content, links, sources, and networks is used to continually improve the relevance and accuracy of the content and sources. The system and method centers on trusted sources of content and applies human intelligence collection techniques to target and exploit trade craft. Persistent and ad-hoc content is incorporated int the human source information network, and content retrieval schemes are integrated into the network data visualization process to provide users with accurate and relevant data in an easily digestible format.

Description

SYSTEM AND METHOD FOR PROVIDING NETWORK SOURCE INFORMATION

FIELD OF THE INVENTION

[0001] The present invention relates to the field of information networks. More particularly, the present invention relates to managing data by managing sources of information to establish source validation and reliability. Further, the invention relates to systems and methods for indexing, matching, linking, sequencing, and collating related sources of information using a network of sources.

BACKGROUND OF THE INVENTION

[0002] Networks and interconnectivity of persons, groups, and organizations has become commonplace. The Internet connects these entities throughout the world by joining billions of connected users as nodes on the network. The World Wide Web and other applications such as instant messaging, electronic mail, blogs, chat rooms, and other peer-to- peer solutions allow direct contact between the nodes. The huge increase in communications capabilities provided by peer-to-peer and other networks has resulted in too much connectivity and too much information, resulting in a cacophony of noise as users attempt to sort quality information from the quantity available. Users want to control accessibility and the information they give and receive. In many cases a particular user would like to limit its reach to smaller subsets of the world- wide Internet community and to limit the information exchanged to ensure reliability, quality, and relevance of the information and content that is provided.

[0003] The amount of information currently available on computer networks such as the World Wide Web is overwhelming. Even an individual knowledgeable in a particular area of research is quickly overwhelmed by the volume and numbers of sources, including human sources and technical sources, that are available on these networks. As a result, users rely on technology to validate information and sources. For example, if a source of information is returned on the first page of a Google™ search, it is instantly given credibility. In this fashion, technology defines what is "expertise" and what is valuable. With the amount of information available on line doubling approximately every two years, the problem of information overload grows more acute.

[0004] During the nascent stages of the Internet, only institutions and individuals with significant financial resources and research capabilities were capable of putting information on the World Wide Web. Thus, the World Wide Web itself was, in fact, a validation tool. With the universality of technology today, this is no longer the case. Anyone, in any part of the world, is capable of placing information and knowledge on-line. There are several problems associated with this ease of adding information. First, the amount of information is enormous, and the corresponding technology can no longer be relied upon to validate information. Second, there are many human sources with incredible knowledge who are not affiliated with institutions or governments who are able to place their knowledge on the Internet, but do not. These sources may lack the resources or the desire to package their knowledge and information in mass media resources or to pay consultants to effectively manipulate the algorithms of search engines to return their posted content. The result is that valuable information is not accessible.

[0005] Additionally, when people define sources on computer networks such as the

World Wide Web, they define the content by the technology such as a Uniform Resource Locator (URL), a search engine, or a database. If a user identifies a good human source of information in an online journal or a blog, the user will continually visit that information location to track that source's knowledge. That human source, however, is not limited by that technology or by that URL or by that database in terms of other areas to which their knowledge is expanding. Good human sources of one particular kind of information are often good sources of information of another kind.

[0006] The vast amount of open source information available on computer networks such as the Internet and the World Wide Web introduces problems for individuals searching for topic-specific information. Users face difficulties locating timely, relevant information. It is often difficult to quickly extract information into a standard and universally usable format, and users have inadequate measures of determining the credibility of the source of information. Search engines such as Google™, undoubtedly help with the first of these issues, yet any given search — especially for a common or generic term (e.g., "terrorism") will return hundreds if not thousands of content links. In addition, the researcher is constantly challenged to locate, digest, and save relevant information, as well as assess the validity of the information presented.

[0007] Identifying quality sources from amongst a large quantity of options remains a challenge. One way to validate the quality of open source information is to identify, examine, and validate the human source that published the information. In short, if the human source is credible, then the information he or she makes available to others may be credible as well. [0008] Efforts to date to improve the accuracy of content available across computer networks have focused on analyzing the content in question with a variety of analysis measures directed to the content itself with little attention focused on improved methods of analyzing and qualifying the source of the content. What is needed is a system and a method for quickly and accurately determining the credibility of sources of information to determine the reliability and precision of the information available.

SUMMARY OF THE INVENTION

[0009] The present invention relates to a system and method for managing data by managing sources of information to establish source validation and reliability. The present invention provides a powerful system and method for identifying, analyzing, indexing, matching, linking, sequencing, and collating related sources of information using a network of sources.

[0010] The system and method of the present invention provides tools that replace chalkboards and tack-boards traditionally used to track human sources in the fields of human intelligence and criminal science. The present invention provides a social network analysis tool (SNA) combined with a robust content management system. Social network analysis tools map and measure relationships and flows between people. The nodes in the network are the people and groups while links show relationships or flows between nodes. The system and method of the present invention draws upon this approach to enable users to search and access reliable information from related sources, whose reliability is assessed by his or her relationships with others and in light of his or her personal history. Rather than simply an SNA tool, however, the system and method of the present invention is used as a roadmap to a robust content base developed around the human sources in the network. [0011] The system and method of the present invention extracts information from a computer network, such as the World Wide Web, into a standardized format for review by analysts. It allows analysts to review and qualify this information as well as information of other origins such as books, newspapers, reports, transcripts, photographic and video images, and the like. This analyst-inputted and vetted data becomes the raw material from which the system and method of the present invention computes the interrelationships between any number of human sources. The system and method of the present invention then provides for the visualization and storage and retrieval of human source data with a mathematical assessment of the source's credibility within the network. [0012] The present invention provides a system and method by which to identify and track human source information networks on computer networks such as the World Wide Web. The system and method of the present invention provides virtual human source networks. Based on established information requirements, a primary human source of information on a particular topic is identified utilizing a number of search capabilities, including biographical, demographic, and profile information. All available expertise and information from the human source, and on the human source, in the form of published documents, reports, interviews, speeches, dialogue in blogs and chat-rooms, and the like, is then tracked on a regular basis and managed under a human source profile. [0013] The information is filtered for relevance. It is summarized, analyzed, and managed with an eye toward the consumer of the information. Analysis is then conducted of the source's relationships such as professional colleagues, research partners, co-authors, and the like, to identify secondary human sources of knowledge of the same subject matter of information and intelligence value. As the network construction progresses, distinct networks within research and knowledge disciplines become apparent, much in the same way people maintain self-limiting social networks. The network mapping process is repeated to establish and maintain a human source information network to be monitored and maintained virtually via the World Wide Web and other computer networks.

[0014] The system and method of the present invention provides users with validated, digestible content. It allows users to easily identify and validate human sources of information via the creation and analysis of source networks and provides users with detailed source profiles for each human source in a given network. Like a roadmap to content, the established links between sources allow researchers to visualize sources in a network and quickly retrieve content gathered for any given source. The system and method of the present invention includes content management features, information vetted by analysts, a customized user experience, a unique searching mechanism, and a one-of-a-kind user interface.

[0015] With the content management feature of the present invention, the user selects a node (that is, an individual human source) in the network and retrieves content surrounding that node such as articles, memberships, conference presentations, blogs, and the like. The system provides a robust content management system that allows content to be collected, assessed, and published. Content may be collected automatically, by web crawlers, for example, or by direct data entry by an analyst. [0016] Additionally, the system and method of the present invention includes information vetted by analysts, including a workstation module that provides analysts with the ability to assess, qualify, and approve information about individuals and relationships. Content is accepted only when a content editor confirms it. Likewise, a relationship between sources is accepted only when a content editor confirms it.

[0017] A user customizes the network generation and has an individualized experience because the user weights or devalues attributes of individuals and relationships to re-draw the network based on criteria that matter to the individual user. Two different users may have the same network with many different links between nodes of the network. User preferences are saved to create a customized page experience for returning users. Recently added sources and recently added content related to recent browsing behavior is presented on a user-specific splash page. The search may be individually iterative and produce results based upon customized search strategies. Additionally, the system constantly collects feedback from users to assess and refine the usefulness of sources and networks. [0018] Further, the system and method of the present invention provides a unique searching mechanism where search terms (for example "Iraqi insurgency") return social networks related to the search term instead of returning simple hits on individual nodes. The system also prompts the user and steers the user toward additional similar networks (for example, "You may be interested in the following related networks"). Additionally, the returned network is generated based on the search term, and does not overwhelm the user with the totality of source relationships for each individual returned by the search term. Further, the system of the present invention includes a state-of-the-art user interface with high usability and visual appeal, including full drag-and-drop manipulation of network diagrams. [0019] The system and method of the present invention delivers operations solutions used to identify and analyze patterns of activity where specific targets or sources are unknown as well as separate instances where specific targets or sources of information are identified and tracked to provide trend information. The present invention profiles sources using available information, collects additional data by extraction, harvesting, and capture techniques, and analyzes the data to determine patterns, to analyze and create links, to create and visualize networks, and to catalog target profiles and associated content. When the initial analysis is complete, the system and method of the present invention employs collaboration techniques to trigger alerts based on trend analysis, provide feedback regarding profiles, links, networks, and content, and to share information throughout the network using messaging techniques. The system monitors itself to improve upon analytic gaps and operations priorities by assessing performance data as well as active and passive feedback. [0020] The system and method of the present invention may include multiple embodiments. For example, in one embodiment, the system and method is provided to organizations with resources to staff a content editor position. In another embodiment, outsourced staffing is provided for operation and maintenance of the system, while in yet another embodiment, the system may be distributed with pre-populated content databases.

GLOSSARY

[0021] The following is a description of some of the terminology and logic used in the system and method of the present invention.

[0022] Source/Node Relevance — the total number of content items that have a match for a given search term.

[0023] Source/Node Prominence — the focus node of a network, the source with most value. Prominence takes into consideration the Relevance, as well as the number of links for a node.

Prominence = (Relevance) x (Number of Children); That is, direct links. Therefore, the prominent node may not have the highest number of related content, but it will have the higher product of content and links.

[0024] Source Size — the size of a node depends on the number of pieces of content that match the search term (Relevance). The more related content, the bigger the source size. [0025] Source Limit — the total appropriate number of sources that are returned per a searched network. In one embodiment, the source limit is set to 300.

[0026] Links Limit — the total (appropriate) number of links between the sources that are returned in a searched network. In one embodiment, the links limit is set to 200. [0027] Overall Link Weight — a measure used to decide which links will be returned for a searched network. It is calculated through the product of three factors:

1. Original weight of the link (the weight that was assigned to the link when it was first created)

2. User's preference for that link type (controlled from Customize Preferences)

3. Link Rating Weight (this a weighted average of the ratings that this link has received. If the link didn't receive any ratings this is defaulted to 1). [0028] Display of Link Types — In one embodiment, the link type is denoted by the color of the line itself that connects 2 nodes. In other embodiments, the type of the link is denoted through the connector that connects the node with a light grey line.

[0029] Display of Source Prominence — In one embodiment, the most prominent source is an orange source, and the rest of the nodes are green. In other embodiments, all nodes are dark grey, and in still other embodiments, the most prominent node is the node that has dark grey color, and the rest of the nodes are light grey. However this color scheme may be altered such that the most prominent node may be in orange, and the rest of the nodes will be in dark grey (visual enhancement).

[0030] Leaf Source — a source that has no other links other that that of its parent.

[0031] Child Source — any source that is directly connected to a previous source is said to be its child.

[0032] Parent Source — any source that precedes a child source and is directly connected to it.

[0033] Descendants — sources that are children of the children of any Parent source all the way until the leaf source.

[0034] Root Source — the most prominent source. This is the source that gets drawn first in a network. It is considered to be the parent of all parent sources.

[0035] Distances between Sources (Borders) — distances between sources are calculated based on formulas that utilize geometric shapes to render the sources in the most appropriate location that prevents overlapping between sources. The main idea is to keep a source and all its children contained (position-wise) within a calculated geometric shape. In some embodiments, the geometric shape that is used is a circle, and the parent is always at the center of its circle. The radius is calculated based on the farthest child away from it.

In other embodiments, both rectangles and circles are used to determine the overall distances.

A rectangle is first used to surround the sources of a parent (which allows the invention to minimize distances between nodes), and then a circle is used to contain the rectangle.

BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings illustrate an embodiment of the invention and depict the above-mentioned and other features of this invention and the manner of attaining them. In the drawings: [0037] FIGURE 1 illustrates relationships between individuals and groups that are mapped into a system in accordance with the present invention to provide an information network for validating sources and information content.

[0038] FIGURE 2 illustrates an example of a platform employed by the system and method of the present invention.

[0039] FIGURE 3 illustrates a method for providing network source information in accordance with the present invention that is broken down into smaller phases.

[0040] FIGURES 4A-4M are a flow diagram illustrating the process for providing network source information in accordance with the present invention.

[0041] FIGURES 5A-5K are graphical versions corresponding to the flow diagram illustrating the process for providing network source information in accordance with the present invention.

[0042] FIGURE 6 illustrates a system for carrying out the process for providing network source information in accordance with the present invention.

[0043] FIGURE 7 illustrates a detail of a user computer for carrying out the process for providing network source information in accordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

[0044] The following detailed description of the invention refers to the accompanying drawings and to certain preferred embodiments, but the detailed description of the invention does not limit the invention. The scope of the invention is defined by the appended claims and equivalents as it will be apparent to those of skill in the art that various features, variations, and modifications can be included or excluded based upon the requirements of a particular use.

[0045] The present invention extends the functionality of information sharing environments and provides a collaborative setting and structure that allows a user community to share insights and knowledge throughout a network of users. The system and method of the present invention identifies sources of information and uses data mining and other information system tools to track sources of information. The system and method of the present invention creates a profile of the source that is used to locate and track other sources of information that are associated with the original primary source of information. In this fashion, information sources may be verified, accredited, and validated. [0046] Potential users of the system and method of the present invention include individuals and groups seeking expert views, including those of medical researchers, health care recipients, and pharmaceutical developers. Additionally, financial service professionals, including analysts, fund managers, insurance brokers, and the like, will also benefit from verified network source information.

[0047] The overabundance of information and the inability of individuals to reliably ascertain the value of the information leaves individuals to their own devices to assess and evaluate information content to determine its usefulness and reliability. There is limited social access to experts including political operatives, intelligence officers, intelligence analysts, and similarly-situated individuals, thereby leaving these individuals and groups without suitable tools to recognize, qualify, and evaluate content sources. Likewise, news reporters, prosecutors, recruiters, students, and any party who has a need for a competitive advantage, employs the system and method of the present invention to recognize expert sources and to efficiently qualify information content. The system and method of the present invention brings disparate information together in a composite picture by methodically scoring, ranking, and sorting information providers and their content. Persons or organizations seeking experts are quickly and efficiently directed to useful and reliable information. The system and method of the present invention diagrammatically produces a network of experts that provides context to the nodes and links of the network. Importantly, the system and method of the present invention incorporates user feedback and content evaluation to get smarter over time by focusing methodologies and processes to highlight useful and validated content while winnowing-out content that users have found to be not valuable. Additional content is also incorporated into the system by evaluating the sources of the new information.

[0048] The information sharing environment of the present invention includes individual sources of information or group sources of information that may be represented as nodes. The individuals or groups are connected to other individuals and groups through various relationships that are represented as links. The relationships may be based upon family, friends, geography, experience, knowledge, education, organizations, interests, goals, governance, and the like. There may be many different types of links between the sources of information (nodes). The number and types of relationships may be characterized and quantified by the system and method of the present invention. The characterization and quantification of the relationships between primary sources of information and secondary sources of information serve to determine the credence paid to the different sources of information. These relationships may be illustrated in a network diagram, where the sources of information (nodes) are drawn as points, and the relationships between the sources of information are drawn as lines.

[0049] An illustration of such a network diagram is illustrated in FIGURE 1. The network depicted in FIGURE 1 illustrates relationships between individuals and groups that are mapped into the system of the present invention to provide an information network for validating sources and information content. The system and method of the present invention leverage the relationships between sources and their content to provide relevant, verifiable content that may be used in applications in a variety of fields as outlined above. The relationships are mapped into a network that describes the relationship and information flow between people, groups, businesses, corporations, and other entities that exist as nodes on the network. Nodes on the network are the physical devices that represent associated entities such as persons, companies, friends, peers, or the like that form a relationship. [0050] The system and method of the present invention utilizes a platform focused on trusted sources of information as a foundation for acquiring valid, reliable insights into establishing a network of information sources to solve business and other information needs. An example illustration of the platform employed by the system and method of the present invention is illustrated in FIGURE 2.

[0051] The method of the present invention may be broken down into smaller phases, such as illustrated in FIGURE 3. The first phase 310 includes the creation of a source profile, the second phase 320 includes data collection, the third phase 330 includes analysis of the collected data, the fourth phase 340 includes collaboration, while phase five 350 performs production and revision to the source and information content.

[0052] Collection Process

[0053] 1. Creating A Source Profile

[0054] As shown in the flow diagram starting with FIGURE 4A, and in the graphical version of the profile creation screen illustrated in FIGURE 5 A, to create a reliable network of information sources, the system and method of the present invention first creates a Source

Profile, beginning with the step of defining information requirements as shown in step 401.

The specifics and the scope of the required information is used to determine appropriate sources, organizations, and suppliers of information. These requirements may also be stored for future retrieval and to use for further iteration once the information sources have been initially determined. [0055] Additionally, in step 403, a user identifies the information collection objectives. The objectives may be ranked or otherwise prioritized to best satisfy the information requirements for the given user and for the given session. Similarly, these objectives may also be stored for future retrieval and to use for further iteration once the information sources have been initially determined.

[0056] Once the information requirements and the information collection objectives are determined and entered, the system and method of the present invention is used to develop a target profile of information sources. In step 405, a user identifies target source candidates of information relevant to collection objectives. In step 407, the user provides detail profile information about targeted sources and documents source profiles. Once the profile information is entered, the system assesses the value of a target source in step 409. The system then, in step 411, evaluates the relevance of target source candidates and compares the relevance to profile requirements and search criteria. This step may include qualifying the relative value of sources based on an objective set of criteria such as access, timeliness, reliability, accuracy, and the like. If the source candidate is found acceptable in step 413, it is selected as a target source candidate in step 417. However, if the source candidate is not found to be acceptable in step 413, the source candidates are filtered against collection objectives and collection methodologies in step 415, and the system again conducts an evaluation process until an acceptable source candidate is determined.

[0057] 2. Data Collection

[0058] The data collection module runs defined processes for managing both persistent and ad hoc information requirements. The module may run automatic or custom content retrieval scheduled at regular search intervals or as required by a particular user in a session. The data collection module may search for content at customizable crawl depths tailored to classified and unclassified host environments anywhere on the web or on internal data stores. Anonymizing and ghosting capabilities may be used for open source web crawls. All file formats are supported by the system and method of the present invention. That is, both structured and unstructured data may be acquired and converted, if necessary, for fast and efficient acquisition and optimization. Additionally, acquired content may be parsed against a customizable list of source names and dictionary terms.

[0059] Once the profile is ready in step 417, data collection activities begin. In step

419, the system and method of the present invention identifies collection sources and methods appropriate for target profiles. The system tasks collection activity to satisfy the information requirements established previously. In step 421, the search criteria is laid out, and in step 423, the content search activities are executed. As also shown graphically in FIGURES 5B 5C, and 5D, additional information and constraints may also be submitted upon executing the data collection search.

[0060] Returning to FIGURE 4C₅ information collection and information processing proceeds in step 425 where the data collected from various collection sources and methods is aggregated, synthesized, and fused to create a single collection entity. In step 427, the collected data is reviewed, and in step 429, it is evaluated to determine the relevance of the collected data for the use at hand. If there is relevant data found acceptable in step 431, it is refined by filtering irrelevant data in step 437. However, if the data collection is not found to be acceptable in step 431 , the collection methods and search criteria are revised in step 433, and the content search is re-executed in step 435 with the new collection methods and search criteria. After the search is re-run, the synthesis, review, and relevance determination steps are performed until the data collection is acceptable in step 431. After filtering irrelevant data in step 437, the relevant data is assigned to source profiles in step 439. [0061] Once the relevant data is assigned to source profiles, the relevance of the collected data is summarized and evaluated in step 441. In step 443, the relative value of the sources based on the collected data is qualified based on a customizable set of criteria. [0062] The criteria used to qualify sources may be based upon a number of different factors. These factors include reporting accuracy, historic reporting reliability, the consistency of source information with other information, and a source's access to information and additional sources. Further, the clarity of source information may also be evaluated and incorporated into the source qualification determination. Likewise, the timeliness of source information, the source knowledge of information requirements, the frequency of source reporting, and trends and patterns in source reporting may also be used to determine source qualification in accordance with the system and method of the present invention.

[0063] Additionally, the system and method of the present invention actively manages sources. For example, analysts may "run" their sources or potential sources by applying tradecraft to sources and associated networks and content and relationships to develop indexes that score, rank, prioritize, and otherwise denote and sort the value of these sources. A full cycle of recruitment and acquisition is applied, including spotting, assessing, developing, and recruiting. Likewise, agent handling, agent communications, and high threat tradecraft environments may also be "run." The result of is a rich targeted analysis, and a feedback loop that leads to the collection of additional, desirable source-based information. [0064] In step 445, the system determines if there are additional target sources of information in the collected data. If all target sources of information are qualified, the process proceeds to step 447 and relevant data is published for analysis, collaboration, and production. When all target sources of information are qualified, the process continues to FIGURE 4F.

[00651 If not all target sources of information are qualified, additional target sources are identified from the collected data in step 449, and profile information about additional targeted sources is detailed in step 451. As additional target sources of information are identified and included, the process continues in FIGURE 4E as additional target source profiles are documented in step 453. Further, the system and method also assesses the value of additional sources in step 455. In step 457 the system and method qualifies the relative value of additional sources based on objective set of criteria or a customized set of criteria that is applicable to the particular session.

[0066] In step 459, the system and method evaluates whether linkages exist between and among documented source profiles. If no linkages exist between source profiles, the relevant data is published for analysis, collaboration, and production in step 467. If, however, linkages exist between source profiles, in step 461 the system and method instantiates these linkages. The linkages may take a variety of forms such as many-to-many, one-to-many, many-to-one, or one-to-one as dictated by the relationship between sources. Once the linkages are instantiated, characteristics that describe dimensions of the source- based relationships are detailed in step 463. Additionally, characteristics that assess the strength of the source-based relationships are also detailed in this step. [0067] In step 465, the relative strength of the source-based relationships is qualified based on objective set of weighting criteria. This relative strength qualification is performed to objectively qualify the strength of the source-based relationships and may employ similar metrics detailed above with regard to qualification of sources, as well as additional network analysis measures.

[0068] Once the linkages and strength of the source-based relationships are determined, this path in the process also proceeds to step 467 to publish relevant, valuable data for analysis, collaboration, and production activities. After the data are published, in step 469, the system and method identifies whether the targeted sources remain relevant and valuable. If, in step 471, the system determines that the targeted sources are no longer relevant, the targeted sources that are no longer relevant or valuable are terminated in step 473. Once the targeted sources that are no longer relevant or valuable are terminated, in step 475, the process determines if any other targeted sources remain. If targeted sources remain, the process continues to step 477 to determine if additional data collection activities on source profiles are needed to satisfy information requirements. If additional data collection activities on source profiles are needed, the process returns to step 419 where collection methods appropriate for the target profile are identified, and the process iterates. [0069] Similarly, if the process in step 471 determines that the targeted sources remain relevant, the process continues with those sources intact and immediately proceeds to step 477 to determine if additional collection activities on source profiles are needed to satisfy information requirements. For example, if recurring web crawls are performed to enhance content retrieval, or if direct website feeds are captured, this additional content is added and the sources are identified and linked. Similarly, customized content and file uploads may be conducted to further utilize additional sources and content. These data feeds may further indicate capabilities of the sources and target source trends. As above, if additional data collection activities on source profiles are needed, the process returns to step 419 where collection methods appropriate for the target profile are again identified, and the process iterates.

[0070] If no additional data collection activities are needed in step 477, the process continues to the analysis portion of the method described below.

[0071] Analysis Process

[0072] 3. Analysis

[0073] The system and method of the present invention includes content management and work flow queues that support source validation and summary analysis. An in-line asynchronous linking module shapes link analysis between sources and forms networks. The targeted, source-based collection of content is populated, parsed, optimized, and examined for source validation. Source names and content watchwords are highlighted and may be color coded for easy review and publication. The analysis module evaluates normal baseline activity and recognizes aberrant activity. The module indicates the target source capability, as well as trends in target source activity. As also shown graphically in FIGURES 5E, 5F, 5G, and 5H, additional information and constraints may also be submitted upon reviewing and refining the analysis. [0074] When no additional data collection activities are required, the system and method of the present invention begins a network search. In step 479, a user enters a keyword or keywords based on information requirements. The keyword(s) are then used to search against the corpus of information. Once the keyword is entered, in step 481 a logic search is performed based on search techniques related to sources, content, and keywords. In step 483, the search returns a network map depicting data visualization of valuable, validated, expert sources that are relevant to the entered keyword(s). Social and professional networks of validated human sources of information are identified through the keyword search function.

[0075] Once the network map is created, a user navigates the network map in step

485 to survey and view sources of interest, including clusters of sources and independent or unlinked sources, from the corpus of information. While navigating the network map, a user may examine links between and among sources of interest, focus on targeted sources that may satisfy information requirements, and review contextual sources that may provide further information regarding the user's search criteria. The user interface module of the present invention presents the network information in a clear and intuitive manner. For example, users may hover their mouse cursor over sources on the network to reveal summary details of profile information. Likewise, users can examine links by hovering over the links to reveal summary details of relationship information. Users may further analyze the network map by examining node sizes, the relative size of which provide further indication as to the relative value of the sources. Similarly, line lengths on the network diagram provide an indication as to the relative strength of the relationships between sources in the network while line colors, indicate the different dimensions of the relationships.

[0076] Once the user has examined the network, the user can determine if the network needs to be refined in step 487. If the network search result is not satisfactory, in step 489 the user may enter additional keyword(s) based on information requirements to further clarify the network search against the corpus. Refining the search allows the user to further focus search criteria to drill-down to the sources, networks and content of greatest relevance. [0077] Once additional keywords are entered, in step 491 the user may perform a refined search using additional keyword logic against the existing network map to return a revised network map, based on search criteria related to sources, content, and keywords. After the refined search is performed, a new network map is returned and link attribute settings render a revised network diagram as in step 483, with the refined network map including valuable, validated, expert sources that are relevant to the refined keyword(s), thereby orienting analysts around a personalized, fully contextualized data visualization experience.

[0078] If, in step 487 the user is satisfied with the network search result, the process continues to step 493 where the user verifies that the link weights and values are satisfactory. If the link attribute weights and values are not satisfactory and need to be adjusted, in step 495 the user defines a link attribute ontology based on dimensions of relationships that are coincident with the information requirements of the session. Once the link attribute ontology is defined, in step 497 the user may re-weigh and value the link attributes against the existing network map to return a revised network map, based on the defined link attribute ontology and social network analysis tools of the present invention related to sources, content, and keywords. After the refined analysis is performed, a new network map is returned and rendered as in step 483, with the refined network map including valuable, validated, expert sources that are relevant to the refined keyword(s).

[0079] Based on the keyword search terms and the networks selected, related networks are identified and presented as well. A user may select from a variety of networks to explore validated and digestible information on the desired topics and keywords. Selected networks, as well as customized networks, can be saved to a User Profile for easy access and retrieval. In this fashion, additional networks may be identified that help visualize content and the context surrounding target sources.

[0080] Once the network search result is satisfactory and the link weights and values are satisfactory, the analysis process continues where the system and method of the present invention analyzes the source profile and the source content. In step 4001, the system analyzes source profile information against information requirements and collection objectives. This includes examination of the source profile background and identifying information, qualifying information that indicates the relative value of the source based on objective set of criteria, qualifying strength and value of source-based relationships based on additional sets of criteria, and related networks within the corpus of which the source may also be a part.

[0081] In step 4003, the system and method of the present invention analyzes source content against information requirements and collection objectives. This analysis includes reviewing the list of relevant data assigned to the source profile that are filtered to display content specific to the network search. Further, the content list is sorted and sifted by category, type, date, content title, and the like. The system and method examines content summaries that may be noted or otherwise flagged for further examination and exploration. [0082] If the source content is satisfactory for display in step 4005, the content is selected and reviewed in step 4011. If the source content is not satisfactory for display in step 4005, the user enters new or additional keywords in step 4007 and re-filters the display content in step 4009 in order to examine additional data assigned to a source profile. The process then returns to step 4003 and the source content is analyzed against the information requirements and the collection objectives until the source content is satisfactory for display in step 4005. Once the content is selected and reviewed in step 4011 , sources are extracted from the content and examined in step 4013. In step 4015, the content is examined and compared to a data dictionary for watchwords. As also shown in FIGURE 5G, watchwords are highlighted in the content.

[0083] Once the content is displayed and reviewed, the system and method of the present invention creates a source network. In step 4017, a name is created for network. In step 4019, the network is saved to a user profile, and in step 4021, the system selects sources relevant to the information requirements of the particular session allowing analysts and users to navigate through a virtual rolodex of existing target profiles of trusted sources of information as well as their most relevant and reliable content. In step 4023, linkages are established between and among source profiles. The linkages may be many-to-many, one-to- many, many-to-one, or one-to-one, depending upon the sources.

[0084] Once the linkages are established, in step 4025, the system and method of the present invention' details characteristics that describe the dimensions of the source-based relationships. In step 4027, the strength of those source based relationships is assessed, and in step 4029, the assessed relative strength of the source-based relationships is qualified based on weighting criteria. Similarly, in step 4031, the value of the source-based relationships is assessed, and in step 4033, the relative value of assessed source-based relationships is qualified based on various criteria as mentioned above.

[0085] Once the relative value of the assessed source-based relationship is qualified, the relevant, valuable data from the created network is viewed for further analysis and production activities in step 4035. The viewed data may then be published in step 4037. The publication of the data from the created network enables group analysis, collaboration, and production activities. When examined by a group, the value and strength of the source-based relationship may be confirmed, or the group may determine that the network is not relevant. If the network is still considered relevant, the process determines if data collection activities on the created network are complete in step 4045. If the network is deemed to no longer be relevant in step 4039, the created network is terminated in step 4041. If, in step 4043, additional created networks remain, the process returns to step 4045 to determine if data collection activities on the created network are complete. If data collection activities on the created network are not complete, the process returns to step 419, and additional collection methods are identified for the target profile. If data collection activities are complete, the process checks to see if that was the final created network to be reviewed in step 4047. If additional networks remain, the process returns to step 489, and additional information is entered with which to create and review additional networks. If, in step 4047, the process determines that the last created network has been reviewed, a user may proceed to a collaboration activity in step 4049, or end their session.

[0086] 4. Collaboration

[0087J If the user chooses to collaborate with other users, they may use a variety of active and passive feedback techniques to pool content and source information within a community of users. As shown in FIGURE 4K, in step 4051, analyst feedback, including ratings and comments regarding the assessed value of target profiles, source-based relationships, content, and networks are recorded. The end users can rate individual sources based upon their credibility and the validity and usefulness of their information. The aggregated ratings are visible for all users to see. Likewise, users may rate overall networks with regard to the responsiveness to the users' requirements, relevance to keywords, or any other criteria deemed useful by the users. Users may also rate individual content based upon the usefulness of the information.

[0088] Once this information is entered and recorded, in step 4053 the feedback is published for analyst community peer and supervisor review, as well as content manager review. A fully integrated feedback queue is employed to manage analyst input regarding content, sources, and networks. With these parties involved in the review process, content may be standardized with regard to format and other structure, and objective criteria may be applied uniformly during the collaborative process.

[0089] In addition to active feedback, passive feedback may also be incorporated as users collaborate. In step 4055, analyst activity on all data transactions is recorded at the application and database level to establish a data log (audit trail) of activity that the analyst performs. Passive feedback review mechanisms and workflow ensure responsiveness to analyst usage patterns. Once this activity is recorded, it may be published in step 4057 for content manager review, as well as administrator review. [0090] In addition to active and passive feedback, the system and method of the present invention may use messaging technology to allow analysts in the community of users to share views on and information gleaned from target profiles, source-based relationships, content, and networks. Analysts may collaborate on production activities related to information requirements, record and publish threaded discussions for further analyst review and follow-up replies, as well as provide future reference points and scheduling milestones with regard to community projects.

[0091] Alerts may also be used to notify analysts of newly developed or newly collected information that may be relevant to their individual or collective information requirements and collection objectives. Content managers may be alerted to the receipt of passive and active feedback, and notification may be provided when reports are produced that detail anomalies, outliers, and trends related to target profiles, source-based relationships, content, and networks. In this manner, the community of users may pool their collective experience, intellect, and opinion to revise and modify the elements of the network source information system and method. Graphical examples of the collaborative menus are illustrated in FIGURES 51, 5J, and 5K.

[0092] 5. Revision and Reiteration

[0093] Once the collaborative work of the analysts and managers is complete, the system and method of the present invention may be used to revise and reiterate through the network of sources to further optimize the system. If a user opts to proceed to production and revision activities in step 4059, they may review passive and active feedback for follow-up response action in step 4061. If a user opts to not proceed to revision activities, they may end their session. In response to the passive and active feedback, in step 4063 a user begins to implement a response. In step 4065, new collection activities are tasked, and currently-tasked collection activities are revised, reprioritized, or terminated. Additionally, in step 4067, target sources are updated, or terminated to ensure information and links are accurate. Similarly, the associated profile information is updated to reflect any changes. [0094] In step 4069, new source linkages are established, and existing source linkages are re-valued, re-weighted, or terminated based upon the analysts' reviews and recommendations. Once the linkages are updated, the link attribute ontology is revised in step 4071. The search logic and methodologies are also tuned and optimized in step 4073 incorporating feedback from users, new relationship and source information, new and updated profile information, and updates to the network. Users may also create their own customized network by selecting individual sources and defining the links between them based on a user's own criteria. This custom network can be saved to a user's personal profile for later use. Metrics to refine and revise search parameters and terms, source directory, content management criteria, and network preferences all provide flexible, intelligent content management. Additionally, users and analysts may provide feedback to highlight collection gaps of information, thereby indicating future requirements and serving to direct resources to fill these gaps.

[0095] In step 4075, the analyst-created networks are published, the watchword dictionary is revised in step 4077, the entity extraction approach is revised in step 4079, and the relationship extraction approach is updated in step 4081. To close the loop regarding production, content queues are purged in step 4083, and active feedback is overridden, if appropriate, in step 4085. The session is then complete.

[0096] The present invention may be implemented by a general purpose computer programmed to accomplish the disclosed functions. Accordingly, the modules described herein may be implemented as computer hardware and/or computer software. Various devices may be used to provide the computer or computer system for effecting the invention.

[0097] System Configuration

[0098] FIGURE 6 illustrates a computer system in which concepts and methods consistent with the present invention may be performed. As shown in FIGURE 6, system 600 includes a number of information sources 101a, 101b, 101c, 101d that may provide or receive content information to and from user's system computer 666. For clarity and brevity, four information sources 101a, 101b, 101c, 101d are shown, but it should be understood that any number of information sources may use the system 600 with which to relay information in accordance with the present invention. Likewise, it should also be understood that any number of user computer systems may be used by the system. For clarity and brevity, a single user computer systems 666 including workstation module 602, linking module 604, analysis module 606, user interface module 608, data collection module 610, assessment module 612, relevance determination module 614, qualifying module 616, and data repository (database) 690 are illustrated in detail in FIGURE 7.

[0099] It should also be understood that information sources 101a, 101b, 101c, 101d and user computer system 666 may be substituted for one another. That is any information source 101a, 101b, 101c, 101d may access information content stored by another information source. In FIGURE 7, user computer 666 is illustrated as modules 602, 604, 606, 608, 610, 612, 614, 616, and 690 merely to show one embodiment and a single configuration. The user computer system 666 can be in a distributed environment, such as servers on the World Wide Web.

[00100] Users 666 may access information sources 101a, 10Ib₅ 101c, 101d through any computer network 198 including the Internet, telecommunications networks in any suitable form, local area networks, wide area networks, wireless communications networks, cellular communications networks, G3 communications networks, Public Switched Telephone Networks (PSTNs), Packet Data Networks (PDNs), intranets, proprietary networks, or any combination of these networks or any group of two or more computers linked together with the ability to communicate with each other.

[00101] As illustrated in FIGURE 6, computer network 198 may be the Internet where information sources 101a, 101b, 101c, 101d are nodes on the network as is user computer 666. Information sources 101a, 101b, 10Ic₅ 101d and user computer 666 may be any suitable device capable of providing information content to another device. For example these devices may be any suitable servers, workstations, PCs, laptop computers, PDAs, Internet appliances, handheld devices, cellular telephones, wireless devices, other devices, and the like, capable of performing the processes of the embodiments of FIGURES 1-7. The devices and subsystems of the embodiments of FIGURES 1-7 can communicate with each other using any suitable protocol and can be implemented using one or more programmed computer systems or devices. In general, these devices may be any type of computing platform connected to a network and interacting with application programs. [00102] While discussed in greater detail with regard to FIGURE 7, user computer 666 identifies and accesses sources of information 101a. 101b, 101c, 101d. User computer 666 sends information requests and receives supplied information content from sources of information 101a, 101b, 101c, 1 Old via computer network 198.

[00103] As illustrated in FIGURE 7, user computer 666 includes workstation module

602, linking module 604, analysis module 606, user interface module 608, data collection module 610, assessment module 612, relevance determination module 614, qualifying module 616, and data repository (database) 690. User computer 666 receives information content from information sources 101a, 101b, 101c, lOld as outlined in the flow diagrams of FIGURES 4A-4M and FIGURES 5A-5K above.

[00104] For example, to establish a source relationship between information sources to establish the accuracy of content provided by the information sources, the data collection module 610 identifies a primary source of information and profiles the primary source of information. The assessment module 612 then assesses a value of the primary source of information, and the relevance module 614 determines a relevance for the primary source of information.

[00105] Once the relevance is determined, the data collection module 610 identifies collection methods with which to collect relevant information from the primary source of information and assigns the relevant information to the profile of the primary source of information.

[00106] The qualifying module 616 then qualifies the value of the primary source of information based upon the collected relevant information. Thereafter, the data collection module 610 identifies secondary sources of information and relationships of the secondary sources of information to the primary source of information based upon the profile of the primary source of information. The qualifying module 616 then qualifies values of the secondary sources of information.

[00107] Once the values are qualified, the linking module 604 instantiates a network of secondary sources of information based upon the qualified value of the secondary sources of information. The analysis module 606 quantifies the relationship between the primary source of information and the secondary sources of information so that the linking module 604 may map linkages embodying relationships between the primary source of information and the secondary sources of information. The analysis module 606 also characterizes content provided by the primary source of information based upon the quantified relationship between the primary source of information and the secondary sources of information. Once the content is characterized, it may be output in a suitable form, such as a report, network diagram, and the like, using workstation module 602 and user interface module 608. [00108] With this configuration, user computer 666 is extremely flexible and responsive to a particular user's needs. For example, a variety of information sources may be used in conjunction with various search requirements and source profiles. Examples of these types of use may be found in APPENDIX A.

[00109] The method of the present invention is reliably fast and accurate, with viable, validated sources providing information content that is the subject of the search query. In addition, the method of the present invention is extremely amenable to data-parallel operations, offering the potential for real-time analysis on highly parallel architectures. [00110] The devices and subsystems of the embodiments of FIGURES 1-7 are for illustrative purposes, as many variations of the specific hardware used to implement the embodiments are possible, as will be appreciated by those skilled in the relevant arts. For example, the functionality of one or more of the devices and subsystems of the embodiments of FIGURES 1-7 can be implemented via one or more programmed computer systems or devices.

[00111] To implement such variations as well as other variations, a single computer system can be programmed to perform the special purpose functions of one or more of the devices and subsystems of the embodiments of FIGURES 1-7. On the other hand, two or more programmed computer systems or devices can be substituted for any one of the devices and subsystems of the embodiments of FIGURES 1-7. Accordingly, principles and advantages of distributed processing, such as redundancy, replication, and the like, also can be implemented, as desired, to increase the robustness and performance of the devices and subsystems of the embodiments of FIGURES 1-7.

[00112] The devices and subsystems of the embodiments of FIGURES 1-7 can store information relating to various processes described herein. This information can be stored in one or more memories, such as a hard disk, optical disk, magneto-optical disk, RAM, and the like, of the devices and subsystems of the embodiments of FIGURES 1-7. One or more databases of the devices and subsystems of the embodiments of FIGURES 1-7 can store the information used to implement the embodiments of the present invention. The databases can be organized using data structures (e.g., records, tables, arrays, fields, graphs, trees, lists, and the like) included in one or more memories or storage devices listed herein. The processes described with respect to the embodiments of FIGURES 1-7 can include appropriate data structures for storing data collected and/or generated by the processes of the devices and subsystems of the embodiments of FIGURES 1-7 in one or more databases thereof. [00113] All or a portion of the devices and subsystems of the embodiments of

FIGURES 1-7 can be conveniently implemented using one or more general purpose computer systems, microprocessors, digital signal processors, micro-controllers, and the like, programmed according to the teachings of the embodiments of the present invention, as will be appreciated by those skilled in the computer and software arts. Appropriate software can be readily prepared by programmers of ordinary skill based on the teachings of the embodiments, as will be appreciated by those skilled in the software art. Further, the devices and subsystems of the embodiments of FIGURES 1-7 can be implemented on the World Wide Web. In addition, the devices and subsystems of the embodiments of FIGURES 1-7 can be implemented by the preparation of application-specific integrated circuits or by interconnecting an appropriate network of conventional component circuits, as will be appreciated by those skilled in the electrical arts. Thus, the embodiments are not limited to any specific combination of hardware circuitry and/or software.

[00114] As stated above, the devices and subsystems of the embodiments of FIGURES

1-7 can include computer readable media or memories for holding instructions programmed according to the teachings of the present invention and for holding data structures, tables, records, and/or other data described herein. Computer readable media can include any suitable medium that participates in providing instructions to a processor for execution. Such a medium can take many forms, including but not limited to, non-volatile media, volatile media, transmission media, and the like. Non-volatile media can include, for example, optical or magnetic disks, magneto-optical disks, and the like. Volatile media can include dynamic memories, and the like. Transmission media can include coaxial cables, copper wire, fiber optics, and the like. Transmission media also can take the form of acoustic, optical, electromagnetic waves, and the like, such as those generated during radio frequency (RF) communications, infrared (IR) data communications, and the like. Common forms of computer-readable media can include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other suitable magnetic medium, a CD-ROM, CDRW, DVD, any other suitable optical medium, punch cards, paper tape, optical mark sheets, any other suitable physical medium with patterns of holes or other optically recognizable indicia, a RAM, a PROM, an EPROM, a FLASH-EPROM, any other suitable memory chip or cartridge, a carrier wave, or any other suitable medium from which a computer can read. [00115] In order to further minimize the overall processing time required to locate, analyze, and evaluate information sources and networks of sources, the system of the present invention may index information content and store these indexed data in the user computer 666. If a user anticipates that they will run the same search query in the future, the user can index the results and store the results. When an indexed and stored information content data set is then executed, the profile, network, content characteristics, as well as display characteristics may be retrieved for those stored results, and the relevant content may simply be updated with additional data that may now be accessible. The information content previously available may be recalled from the user computer 666 to reduce the overall execution time.

[00116] The foregoing description of the aspects and embodiments of the present invention provides illustration and description, but is not intended to be exhaustive or to limit the invention to the precise form disclosed. Those of skill in the art will recognize certain modifications, permutations, additions, and combinations of those embodiments are possible in light of the above teachings or may be acquired from practice of the invention. Therefore, the present invention also covers various modifications and equivalent arrangements that fall within the purview of appended claims.

APPENDIX A

Industry Solutions

Recruiting Solutions

-Framework for collecting and analyzing recruiting information

-Targeted sourcing and selection evaluation support

-"Run" virtual recruitment source networks

-Built-in methodology for information validation

-Manage knowledge, skill, ability, and attribute factors

-Candidate and source profile details, targeting and tracking information

-Validated content management services, including indexing/filtering/scoring

-Interactive and collaborative information sharing environment; feedback and message center

-Integrated active and passive feedback mechanisms to ensure responsiveness and improve intelligence value over time

-Supports data discovery, data visualization, and advanced analytic techniques

-Performance metrics on skill gaps, sourcing effectiveness, and requisition fulfillment

-Suggests alternative recruitment pursuits

-Social and network link analysis connects sources for quality candidates

-Federated search capability fuses data by scanning all available internal/external data sources

Staffing Solution

-Framework for collecting and analyzing strategic staffing information

-Strategic staffing support

-"Run" virtual staffing networks

-Built-in methodology for information validation

-Manage teamwork efficiency and performance evaluation factors

-Staff and team profile details, targeting and tracking information

-Validated content management services, including indexing/filtering/scoring

-Supports data discovery, data visualization, and advanced analytic techniques

-Performance metrics on resource gaps, workload balancing, and optimal team mix

-Suggests alternative staffing pursuits

-Social and network link analysis connects effective staff mixes

Legal. Financial Planning. Mergers and Acquisitions., and Market Research and Analysis

-HBl Visa research

-Applied legal research

-Pre-trial/discovery

-Expert witness identification/validation National Security Solution

-Framework for collecting and analyzing all-source intelligence information

-"Run" virtual human source networks

-Built-in methodology for information validation

-Asset tracking and source targeting

-Manage collection objectives

-Source profile details

-Validated content management services, including indexing/filtering/scoring

-Data discovery, data visualization

-Supports "what-if ' analysis, hypothesis testing, evidence chaining, and other advanced analytic techniques

-Performance metrics on collection gaps, analytic activities, and requirements fulfillment

-Suggests alternative collection and analysis pursuits

-Social and network link analysis "connects the dots"

-Federated search capability fuses data by scanning all available internal/external data sources, high/low side friendly

-Voice Capture

Pharmaceutical Solution -Competitive intelligence —Product release -Clinical trials -Peer review

Health Care Solution -Competitive intelligence —Care from perspective of consumers -Research support for doctors/care -Applied medical research

Health Insurance -Corporate risk analysis -Consumer research for coverage APPENDIX B

[00117J User Interface Graphic Design

[00118] The visualization of social networks is as important as the rules that establish them. Visualization strategies must account for how viewers are likely to decode the information presented to them as well as addressing the quality and quantity of information encoded within a network graph. Complexity is more likely to introduce unintended information. Building intended consequences directly into the layout with clear visual indicators is a good idea, but it does not in itself control unintended consequences, which should be balanced with the other considerations of visual display.

[00119] The following general rules are used in the system and method of the present invention:

[00120] 1. The fewer the number of lines crossing, the better the network diagram.

[00121] 2. Utilization of different colors and shapes for nodes and links is helpful to the user.

[00122] 3. Motion and presentation of sub-networks is useful under certain circumstances (for example, motion can be useful in displaying network changes over time.)

[00123] 4. The weight of the relationship is represented by line distance, and the relative importance of the individual in the network is represented by node size.

[00124] General Navigational Features

[00125] When the system and method of the present invention provides links to third- party resources, the system will serve a warning message that the system is not responsible for content on those third-party sites.

[00126] The system and method of the present invention implemented a global navigation bar that provides links back to the home page or key sites (e.g., an analyst work bench). This bar is placed in a narrow strip at the top of all web sites.

[00127] The system and method of the present invention does not use pop-up windows to serve information to site visitors. The menu items on the top navigational bar will display links to the corresponding additional sections on mouse over.

[00128] The system and method of the present invention includes a breadcrumb trail on all site pages so that users can identify where they are on the site and return to previous pages. 100129] User Access Roles

[00130] Each user has a unique login ID and password along with an associated permission level.

[00131] 1. Analysts have the ability to use an Analyst Work Bench to view interactive networks and other related resources. Analysts also have content submitter functionality.

[00132] 2. Content Submitters have the ability to submit information to be approved by a manager or an administrator and added to the user database. Submitters also have analyst functionality.

[00133] 3. Content Managers have all functionality of submitters and analysts, as well as the ability to mitigate certain relationships between data; add, edit, and delete content; and publish submitter content.

[00134] 4. Administrators have all functionality of managers, submitters, and analysts, as well as control over user access. Administrators have the ability to create, edit or delete users in the system. The system and method of the present invention provides administrators with the ability to generate a report on users and associated permission levels

[00135] User Workflows and General Site Features

[00136] 1. Home Page — This page includes static content, but the system allows

Administrators and Content Managers to edit the welcome text and post announcements. [00137] 2. Search Engine — A search field is available on every page of the site that searches on the following source profile elements: a. Title b. Body c. Special Keywords

[00138] 3. Log — The system will log a history of changes to source profiles, including: a. Date of Change b. Username of editor c. List of fields that were edited

[00139] 4. Status — The system will maintain a status of all content, including: a. Live b. Pending c. Rejected [00140] 5. Email notification — The system will email every member of the "Manager" group when new content is submitted for approval (OR pending content will appear when a manager or admin logs in). The system will email the content submitter when content is rejected or approved. Rejected content will be filed as rejected; pending as pending.

[00141] Content Submission — On the "Submit Content" page users will choose the type of content to submit from a dropdown list of options: a. Source profile b. Document c. Link d. Image

[00142] Users will be able to search to see if a given source profile already exists. The system will assign each source profile a unique identification number. The system allows a single source to have multiple aliases associated with its source profile (e.g. If a source is referenced as Daniel Fielding, Daniel J. Fielding, and DJ. Fielding in publications, then his profile will have three aliases.) The system allows multiple sources to have the same name (e.g. If there are two different sources with the name "Daniel Fielding").

[00143] Analyst Workbench — includes the following features: a. My Toolbar

1. My Rules

2. My Options b. News Ticker c. Network Map

1. Localized View

2. Global Perspective d. Related Resources e. Provides links to relevant information

1. Document information will be displayed in standard MLA format f. Resource Lists may be exported in word format

Claims

THE CLAIMED INVENTION IS:

1. A method of establishing a source relationship between information sources to establish the accuracy of content provided by the information sources, the method comprising: identifying a primary source^" of information; profiling the primary source of information; assessing a value of the primary source of information; determining a relevance for the primary source of information; identifying collection methods with which to collect relevant information from the primary source of information; assigning the relevant information to the profile of the primary source of information; qualifying the value of the primary source of information based upon the collected relevant information; identifying secondary sources of information and relationships of the secondary sources of information to the primary source of information based upon the profile of the primary source of information; qualifying a value of the secondary sources of information; instantiating a network of secondary sources of information based upon the qualified value of the secondary sources of information; quantifying the relationship between the primary source of information and the secondary sources of information; mapping linkages embodying relationships between the primary source of information and the secondary sources of information; and characterizing content provided by the primary source of information based upon the quantified relationship between the primary source of information and the secondary sources of information.

2. A system for establishing a source relationship between information sources to establish the accuracy of content provided by the information sources, the system comprising: a data collection module that identifies a primary source of information, profiles the primary source of information, and identifies collection methods with which to collect relevant information from the primary source of information and that further identifies secondary sources of information and relationships of the secondary sources of information to the primary source of information based upon the profile of the primary source of information; an assessment module that assesses a value of the primary source of information; a relevance determination module that determines a relevance for the primary source of information and assigns the relevant information to the profile of the primary source of information and that quantifies the relationship between the primary source of information and the secondary sources of information; a qualifying module that qualifies the value of the primary source of information based upon the collected relevant information and qualifies a value of secondary sources of information; a linking module that maps linkages embodying relationships between the primary source of information and the secondary sources of information and that instantiates a network of secondary sources of information based upon the qualified value of the secondary sources of information; an analysis module that characterizes content provided by the primary source of information based upon the quantified relationship between the primary source of information and the secondary sources of information; and a workstation and user interface module that outputs the characterized content to a user.

3. A data storage medium with computer-executable instructions for rendering a contour graphic region for display, the data storage medium comprising: instructions for identifying a primary source of information; instructions for profiling the primary source of information; instructions for assessing a value of the primary source of information; instructions for determining a relevance for the primary source of information; instructions for identifying collection methods with which to collect relevant information from the primary source of information; instructions for assigning the relevant information to the profile of the primary source of information; instructions for qualifying the value of the primary source of information based upon the collected relevant information; instructions for identifying secondary sources of information and relationships of the secondary sources of information to the primary source of information based upon the profile of the primary source of information; instructions for qualifying a value of the secondary sources of information; instructions for instantiating a network of secondary sources of information based upon the qualified value of the secondary sources of information; instructions for quantifying the relationship between the primary source of information and the secondary sources of information; instructions for mapping linkages embodying relationships between the primary source of information and the secondary sources of information; and instructions for characterizing content provided by the primary source of information based upon the quantified relationship between the primary source of information and the secondary sources of information.