JP2011529600A - Method and apparatus for relating datasets by using semantic vector and keyword analysis - Google Patents

Abstract

  The present disclosure describes a web that is being reviewed by a user based on an analysis of a unique semantic vector, such as a trainable semantic vector (TSV) representing web pages and advertisements, and information on representative keywords of advertisements and web pages. Systems and methods are described for identifying one or more data sets, such as advertisements, that are contextually related to a target data set, such as a page.

Description

FIELD OF DISCLOSURE The present disclosure relates to methods and systems for identifying contextually related datasets such as documents, web pages, emails, search queries, advertisements, and more particularly to the unique meaning of a dataset. The present invention relates to a method and system for identifying a data set contextually related to a target data set by analyzing a keyword semantic expression including information on vectors and representative keywords of the data set.

Background and Summary of this Disclosure Microsoft Corporation, Google Inc. , Vibrant Media or Yahoo! Inc. Search engines or ad placement systems are widely used, such as those developed by, to identify documents or files that are potentially relevant to a search query input by a user, or documents, email messages, RSS feeds, web pages Select and display advertisements that are viewed or manipulated by the user, or contextually associated with one or more data sets being viewed or manipulated, such as

  However, existing search engines or ad placement systems have been developed and modified several years ago and are still far from satisfactory. Search results or identification queries often lack sufficient relevance to the search query entered by the user, or to the document or web page being viewed or viewed by the user.

  The present disclosure provides contextual analysis to a target dataset such as a search query or a web page viewed by a user by analyzing a semantic expression that includes information on a unique semantic vector representing the dataset and representative keywords of the dataset. Various embodiments are described that efficiently identify one or more data sets, such as documents, web pages, emails, etc., that may be relevant.

An exemplary method according to the present disclosure controls a data processing system for associating at least one data set from a group of data sets with a target data set. Each data set or target data set includes at least one keyword. The method accesses a semantic vector representing the target data set and a respective semantic vector representing each respective data set in the group. Each semantic vector representing each respective data set in the group is a collection of relationships between each at least one keyword in the respective data set and a predetermined category to which each at least one keyword in the respective data set can be related. Contains information. The semantic vector representing the target data set includes collective information of a relationship between each at least one keyword in the target data set and a predetermined category to which each at least one keyword in the target data set may be related, and the target data set Or a semantic vector representing each respective data set in a group has a dimension equal to the number of predetermined categories. For each data set in the group, a first comparison between the target data set and each data set in the group is made by comparing the semantic vector associated with the target data set with the semantic vector associated with each data set in the group. Determining the similarity of. The exemplary method further accesses the keyword semantic representation of the subject dataset and the keyword semantic representation of each respective dataset in the group. The keyword meaning expression of the target data set or the keyword meaning expression of each data set in the group includes information representing the representative keyword of each data set in the target data set or group, and the keyword meaning expression or group of the target data set The keyword semantic representation of each respective data set in is constructed differently from the semantic vector of the target data set or the semantic vector of each respective data set in the group. For each data set in the group, a second similarity between the target data set and each data set in the group by comparing the keyword semantic representation of the target data set with the keyword semantic representation of each data set in the group Determining gender. At least one of the data sets in the group is selected according to a first similarity between the target data set and each data set in the group and a second similarity between the target data set and each data set in the group. The method relates at least one selected data set in the group to the target data set. At least one of the data sets can be presented to the user at the same time as the subject data set, or can be presented after presenting the subject data set to the user. At least one of the data sets or the target data set may be presented to the user in an auditory form, a visual form, a video form, a tactile form, or any combination thereof.

  In one embodiment, at least one of the data sets in the group is an advertisement, and the target data set is a document, web page, email, RSS news feed, data stream, broadcast data or information about the user or one or more documents. , Web pages, emails, RSS news feeds, data streams, broadcast data or part of user information. According to yet another embodiment, the exemplary method communicates to the user at least one selected data set or a file associated with or associated with the selected data set for the target data. Can do. Displaying at least one selected data set, playing an acoustic signal in accordance with at least one selected data set, or providing a link to at least one selected data set; The selected data set can be communicated to the user.

  In one embodiment, the at least one keyword is a word, phrase, character string, pre-assigned keyword, sub-data set, meta-information, and information retrieved based on links included in each data set. At least one of the following. In another embodiment, the semantic vector for each data set is pre-calculated and included in each data set. Semantic vectors can be generated dynamically on the fly.

  According to one embodiment, the semantic vector representing each respective dataset in the group is between at least one keyword of each respective dataset in the group and a predetermined category to which a known keyword and a known keyword can relate. A semantic vector that is constructed based on the known relationships of and represents the target data set is a known relationship between at least one keyword of the target data set and a predetermined category to which the known keyword and the known keyword can relate. Built on the basis. According to another embodiment, the semantic vector associated with each data set is further generated based on information about at least one user or at least one data set linked to each data set. Information about at least one user may include at least one of previously viewed documents, previous search requests, user preferences, and personal information.

According to one embodiment, the data sets in the group according to a first similarity between the target data set and each data set in the group and a second similarity between the target data set and each data set in the group. Selecting at least one of the first similarity and the second similarity as a primary similarity and the other as a secondary similarity, and a plurality of primary similarity Accessing information of a preset association level, mapping, for each dataset in the group, a primary similarity to one of the preset association levels according to the primary similarity, and mapping the dataset in the group A rating that ranks the datasets in the group according to each pre-set relevant level And within each association level, ranking the dataset to each association level according to the secondary similarity of the dataset, and at least one of the datasets in the group according to the results of ranking the dataset to each association level. Selecting.

  According to another embodiment, the datasets in the group according to a first similarity between the subject dataset and each dataset in the group and a second similarity between the subject dataset and each dataset in the group. Selecting at least one of the first similarity and the second similarity as primary similarity and the other as secondary similarity, and according to the primary similarity, the data in the group Ranking the set; selecting at least one candidate data set from the graded data set according to preset criteria; ranking at least one candidate data set according to secondary similarity; and at least one According to the results of ranking the two candidate datasets. And selecting at least one of Tasetto.

  According to yet another embodiment, the data in the group according to a first similarity between the target data set and each data set in the group and a second similarity between the target data set and each data set in the group. The step of selecting at least one of the sets comprises, for each data set in the group, a composite similarity according to a preset formula based on the respective first similarity of the data set and the second second similarity of the data set. And calculating at least one of the data sets in the group according to the respective composite similarity of the data sets.

Another aspect of the invention is an exemplary data processing system for associating at least one data set from a group of data sets with a target data set. Each data set or target data set includes at least one keyword. The system includes a data processor configured to process data and a data storage system configured to store instructions that control the data processor to perform specified steps when executed by the data processor. The steps include accessing a semantic vector representing the target data set and a respective semantic vector representing each respective data set in the group: each semantic vector representing each respective data set in the group The semantic vector representing the target data set includes collective information of a relationship between each at least one keyword in the data set and a predetermined category to which each at least one keyword in the respective data set may be related. The collective information of the relationship between the at least one keyword in and the predetermined category to which each at least one keyword in the target data set may be related, representing the respective data set in the target data set or group Kuttle has a dimension equal to the number of predetermined categories, and for each dataset in the group, by comparing the semantic vector associated with the target dataset to the semantic vector associated with each dataset in the group Determining a first similarity between the target dataset and each dataset in the group; accessing a keyword semantic representation of the target dataset and a keyword semantic representation of each respective dataset in the group; The keyword meaning expression of the target data set or the keyword meaning expression of each data set in the group includes information representing the representative keyword of each data set in the target data set or group. The word semantic representation or keyword semantic representation of each dataset in the group is constructed differently from the semantic vector of the target dataset or the semantic vector of each dataset in the group, and for each dataset in the group Determining a second similarity between the target data set and each data set in the group by comparing the keyword semantic representation of the target data set with the keyword semantic representation of each data set in the group; At least one of the datasets in the group according to a first similarity between the dataset and each dataset in the group and a second similarity between the subject dataset and each dataset in the group; Step to choose And associating at least one selected data set in the group with the target data set.

  The example systems described herein may be implemented using one or more computer systems and / or appropriate software.

  Embodiments herein control a data processing system to perform machine-executed steps during execution of the data processing system to relate at least one data set from a group of data sets to a target data set. Includes machine-readable media for carrying instructions. Each data set or target data set includes at least one keyword. The step includes accessing a semantic vector representing the target data set and a respective semantic vector representing each respective data set in the group, wherein each semantic vector representing each respective data set in the group is a respective one. The semantic vector representing the target data set includes collective information of a relationship between each at least one keyword in the data set and a predetermined category to which each at least one keyword in the respective data set may be related. The collective information of the relationship between each at least one keyword in and the predetermined category to which each at least one keyword in the target data set may be related, and represents each respective data set in the target data set or group The vector has a dimension equal to the number of predetermined categories, and for each dataset in the group, the semantic vector associated with the target dataset is compared with the semantic vector associated with each dataset in the group. Determining a first similarity between the target data set and each data set in the group, and accessing a keyword semantic representation of the target data set and a keyword semantic representation of each respective data set in the group The keyword meaning expression of the target data set or the keyword meaning expression of each data set in the group includes information representing the representative keyword of each data set in the target data set or group, and the target data set Keyword semantic representation of each data set in the group or keyword semantic representation of each data set in the group is constructed differently from the semantic vector of the target data set or the semantic vector of each respective data set in the group. Determining a second similarity between the target data set and each dataset in the group by comparing the keyword semantic representation of the target dataset with the keyword semantic representation of each dataset in the group for the set; At least one of the data sets in the group according to a first similarity between the target data set and each data set in the group and a second similarity between the target data set and each data set in the group Select And associating at least one selected data set in the group with the target data set.

Additional advantages and novel features of the present disclosure will be set forth in part in the description which follows, or in part will be apparent to those skilled in the art from the following discussion, or may be learned by practice of the disclosure. The illustrated and described embodiments provide an illustration of the best mode contemplated for carrying out the disclosure. Each feature and embodiment described herein may be implemented alone or in combination with other features or embodiments. The present disclosure can be modified in various obvious aspects without departing from the spirit and scope thereof. The drawings and description are to be regarded as illustrative in nature and not as restrictive. The advantages of the disclosure may be realized and attained by means of the instrumentalities and combinations particularly pointed out in the appended claims.

1 is a block diagram of an exemplary advertisement placement system. 1 illustrates an example advertisement placement system embodiment in accordance with the present disclosure. 6 illustrates the operation of another embodiment of an advertisement placement system according to the present disclosure. 3 is an exemplary table showing relationships between words and categories. 5 is an exemplary table illustrating values corresponding to the importance of words from FIG. FIG. 5 is an exemplary table illustrating a representation of words from FIG. 4 in a semantic space. FIG. 2 is a block diagram of an example computer system in which an example advertisement placement system is implemented.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS The present disclosure is illustrated by way of example in the accompanying drawings and not by way of limitation. In the drawings, elements having the same reference number designation represent the same element throughout the drawings.

  In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the disclosed concepts may be practiced or implemented without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present disclosure.

  As used in the description herein, the term “data set” refers to a collection of expressions that are readable and / or understandable by humans and / or machines, and the term “keyword” Indicates one or more elements such as symbolic elements, numbers, etc. For example, if the data set is a document, the keywords can be one or more words, phrases, punctuation marks, symbols and / or sentences included in the document. The data set can be a collection of different types of data sets or part of a larger data set. A data set may be a summary and / or tag that summarizes or describes the contents of another data set. The keywords may or may not be directly visible to the user. For example, the keywords can be part of a metadata element associated with a subtitle or hidden subtitle of a video file, lyrics of an audio file, or a Word document. Additional processing may be performed before the keyword can be confirmed or processed by a person or machine. For example, optical character recognition or speech recognition may be used to convert certain elements in a first format to a second format for easier processing and / or recognition by a person or machine.

  Examples of datasets are web pages, video, audio or multimedia files, advertisements, emails, documents, RSS feeds, multimedia files, photos, diagrams, drawings, electronic computer documents, recordings, broadcasts, video files, metadata Or including one or more of the above.

Examples of keywords include words, phrases, symbols, terms, hyperlinks, metadata information and / or any displayed or hidden items that are included or associated with the dataset. In the context of this disclosure, a “web page” is a Microsoft Internet Explorer (this content may include, but is not limited to, HTML pages, Java® Script pages, XML pages, email messages, and RSS news feeds). Is understood to indicate any combination or collection of information that can be displayed in a web browser.

  As used in this disclosure, the term “target dataset” refers to the exemplary system identifying one or more datasets that are contextually related to the target dataset from a group of datasets. Indicates one or more data sets to be attempted. For example, the subject data set seeks to find a search query entered by the user with the intent of finding documents related to the search query, or an advertisement suitable for display by the exemplary system according to the present disclosure with a web purge. Can be one or more web pages.

  For illustrative purposes, the following example shows a unique semantic vector, such as a trainable semantic vector (TSV) that represents web pages and advertisements, and a semantic representation that includes information about representative keywords for advertisements and web pages. The operation of an embodiment that identifies one or more data sets, such as advertisements contextually related to a target data set, such as a web page being reviewed by a user, based on the analysis is described. Various formulas and statistical operations can be performed to identify important or representative keywords so that they can be valued more than others.

  It will be appreciated that similar approaches and methodologies may be applied to different types of data sets and / or subject data sets. For example, using a similar approach, identify documents or web pages that are contextually related to one or more prosecution queries (target datasets) entered by the user, or have potential for one or more advertisements It is possible to identify web pages that may be relevant.

  A trainable semantic vector (TSV) is a unique type of semantic representation of a data set and is generated based on the data points contained in the data set and the known relationships between known data points and a predetermined category. Details of the construction and properties of the trainable semantic vectors are filed on May 2, 2000, the disclosure of which is hereby incorporated by reference in its entirety, and is entitled “CONSTRACTION OF TRAINABLE SEMANTIC VECTORS AND CLUSTERING”. No. 6,751,621, and US Patent Application No. 11 / 126,184 filed May 11, 2005 and entitled ADVERTISEMENT PLAMETENT METHOD THE SYSTEM USING SEMANTIC ANALYSIS (Attorney Docket No. 55653-019) It is described in.

  FIG. 1 is based on an analysis of at least two types of semantic representations of a group of advertisements 12 and web pages 11, that is, a semantic representation including information on representative keywords of TSVs and advertisements 12 and web pages 11. 1 is a diagram of an example advertisement placement system 10 configured to identify one or more advertisements that are contextually related to a web page 11 being viewed by a user from advertisements 12. The advertisement 12 can consist of any combination of media such as text, sound or animation. Based on the results of the analysis, the system 10 generates a matching result that identifies the selected advertisement that is contextually related to the web page 12.

Selection of one or more advertisements for a particular data set or web page can occur when the data set is presented, or after or before the data set is presented to the user. In another embodiment, the advertisement placement system 10 is used to contextually associate a web page 11 such that the web page 11 is displayed with or linked to the one or more selected advertisements. One or more advertisements 12 are selected. The data set identified to be associated with the target data set may be communicated or presented to the user along with the target data set, or may be communicated or presented at a different time than the presentation or transmission of the target data set. The data set may be communicated or presented to the user in various forms or formats such as auditory form, video form, visual form, tactile form, machine readable form or any combination thereof.

  The TSV associated with each advertisement 12 or web page 11 may be calculated in advance or calculated on the fly. In one embodiment, each web page or advertisement includes embedded or associated information of their respective pre-calculated TSVs. In another embodiment, the TSV associated with the web page 11 is dynamically calculated by the system 10.

  FIG. 2 is a detailed block diagram of an embodiment of the advertisement placement system 10. As shown in FIG. 2, the advertisement placement system 10 includes term extractors 102, 112 for identifying and retrieving keywords from advertisements 12 or web pages 11. The term extractors 102, 112 perform a linguistic analysis on the content of the advertisement 12 or web page 11 and divide the sentence from the advertisement 12 or web page 11 into smaller units such as words, phrases, and the like. Frequently used terms such as grammatical words such as “the”, “a”, etc. can be removed using a preset stoplist. If the advertisement 12 or web page 11 contains information other than the actual content (eg, HTML markup tags or Java® Scripting), that information can be removed. Software for performing term extraction is widely available and known to those skilled in the art.

  The advertisement placement system 10 further includes TSV generators 103, 113 for calculating a TSV for the web purge 11 or advertisement 12 based on the output from the term extractors 102, 112. System 10 may use a common TSV generator for both advertisement 12 and web page 11. Alternatively, separate TSV generators may be used to process the output from web page 11 and advertisement 12, respectively.

  The ad placement system 10 includes a TSV indexer 114 and a TSV index database 118 that are used to organize and store TSVs generated for efficient searching. The TSV indexer 114 may be implemented using a full database management system (DBMS) or simply a software package for managing large data records, and the TSV index database 118 is a TSV that contains the TSV of the advertisement 12 with that link. It can be implemented with a database that stores the index file. Different indexing schemes can be applied to speed up the search. For example, one common indexing scheme for TSVs lists them under the individual semantic categories referenced by the TSV.

The TSV associated with each advertisement 12 and the TSV associated with the web page 11 are input to the TSV matcher 104 to determine the respective TSV similarity between the web page 11 and each advertisement. Similarity can be in the form of a related score. In one embodiment, the similarity or association between TSVs is determined based on the distance between semantic vectors (TSVs), such as determining the N-dimensional Euclidean distance between TSVs. N is the number of dimensions of the semantic space or the predetermined category. The shorter the distance between the TSV of the web page 11 and the TSV of the advertisement, the more similar between the web page 11 and the advertisement. Other comparison methods such as cosine measure, Hamming distance, Minkowski distance or Mahalanobis distance can also be used. Various optimizations can be performed to improve the comparison time, including reducing the dimension of the TSV prior to the comparison or applying a filer to eliminate certain advertisements before and after the comparison. .

  Based on the TSV comparison results, the TSV matcher 104 generates a TSV match list 105 that includes a rating list of matched advertisements selected from the advertisements 12 according to their respective TSV similarities with the web page 11. A preset threshold may be applied to select only those advertisements that have a similarity that exceeds the preset threshold.

  The advertisement placement system 10 further includes a mechanism for determining and comparing contextual expressions that have a different type than the TSV for the web page 11 and the advertisement 12. In one embodiment, the advertisement placement system 10 generates a semantic expression that includes information about the representative keywords of the web page 11 and the advertisement 12.

  As shown in FIG. 2, the keyword selectors 115, 106 input the terms retrieved by the term extractors 102, 112, the term frequency (how often the term occurs on the page), the inverse document frequency (one collection). From the content of the web page 11 or advertisement 12 to represent the web page 11 or each advertisement 12 according to one or more metrics such as other approaches known to those skilled in the art Select a subset of keywords. For example, the keyword selectors 115 and 106 may calculate the appearance frequency or the number of appearances of each text of the web page 11 or each advertisement, and select a representative keyword based on the calculated appearance frequency or the number of appearances of each text. .

  Another example is to use a stoplist to remove keywords that provide less information about the subject of the web page 11 or advertisement 12. The term extractors 102, 112 maintain or access a stoplist that includes the most commonly occurring words that provide little information about the subject. Keywords included in the stoplist are not good search terms. The stoplist can be created by a language specialist, by automatic analysis (such as statistical), by the user, or by a combination of all three. It will be appreciated that other approaches known to those skilled in the art may be used to select keywords from web page 11 or advertisement 12 to represent web page 11 or advertisement 12.

  After the representative keyword of each advertisement is identified by the keyword selector 115, a keyword index database 117 is provided to store the representative keyword and links to the respective advertisement 12.

  As shown in FIG. 2, a keyword matching unit 107 is provided, and the keyword similarity between the web page 11 and each advertisement 12 based on the information of each selected advertisement and the selected keyword representing the web page 11. Determine sex. In one embodiment, the keyword matcher 107 examines the selected set of keywords for the web page 11 in the keyword index database 117 and searches for the keyword for each advertisement and web page 11 according to one or more known algorithms. Generate a related score. For example, a relevance score between two sets of representative keywords is calculated based on the number of matching or common keywords (one term, one vote) included in advertisements and web pages. In another embodiment, the keyword matcher 107 is a more sophisticated voting scheme (electoral group, weighted share, aristocratic with absolute veto, roundness of support). Is used to determine the similarity between each advertisement and web page 11. Other types of calculations, such as a vector space model, can be obtained by using a straight or modified cosine similarity measure (straight or modified cosine similarity measure). Can be used to calculate a related score.

  The keyword matching unit 107 calculates the respective similarity between the web page 11 and each respective advertisement, and then ranks the advertisement 12 based on the respective similarity with the web page 11 and each related score. The keyword matching list 108 to be generated is generated.

  The TSV matching list 105 and the keyword matching list 108 are transmitted to the combiner 109 that generates the final matching list 110 according to the information included in the keyword matching list 108 and the TSV matching list 105. In one embodiment, for each advertisement in TSV match list 105 or keyword match list 110, combiner 109 calculates a composite relevance score based on its relevance score in TSV match list 105 and keyword match list 110. A final match list 110 is then generated according to each composite association score for each advertisement.

In one embodiment, a composite association score is calculated as follows:
If the advertisement is included in both the TSV match list 105 and the keyword match list 108,
Composite score = a ₁ * TSV score + b ₁ * keyword score + c ₁ (1)
If the advertisement is only included in the TSV match list 105,
Composite score = a ₂ * TSV score + c ₂ (2)
If your ad is only on the keyword match list 108,
Composite score = b ₃ * Keyword score + c ₃ (3)
The coefficients a ₁ , a ₂ , b ₁ , b ₃ , c ₁ , c ₂ , c ₃ can be selected so that Equation 2 and Equation 3 are special cases of Equation 1. Relevance scores in any or all of the match lists can be normalized to [0, 1]. Conditional or unconditional thresholds can be matched to relevant scores in any or all matched lists to shorten the list. The final match list 110 is compiled according to the composite score of the advertisement.

  In another embodiment, advertisements in the TSV match list 105 and keyword match list 108 are rearranged to form an exemplary final match list 110 using a unique formula. Each advertisement in the TSV matching list 105 and the keyword matching list 108 is associated with a respective TSV related score and keyword related score. The TSV matching list 105 ranks advertisements according to their respective TSV-related scores, and the keyword matching list 108 ranks advertisements based on their respective keyword-related scores. One of the TSV related score and the keyword related score is designated as a primary related score, and the other is designated as a secondary related score.

  Table 1 shows an exemplary rating list with a TSV related score as a primary related score and a keyword related score as a secondary related score. For illustration purposes, the associated score is normalized to [0, 1].

  The primary relevant score for each advertisement is mapped to a preset relevant level corresponding to a specific range of relevant scores. The advertisements are then rated according to their mapped relevance level. The secondary relevance score for each respective ad is used to rate the ad within each relevance level.

  For example, in the example shown in Table 1, the TSV association score is mapped to three different association levels.

If the related score is <0.4,
Relevance level = 1.

If 0.4 <= relevant score <0.7,
Relevance level = 2.

If the related score> = 0.7,
Association level = 3.

  After conversion, the advertisements are rerated according to their respective associated levels. Ads within each respective relevance level are then rerated according to their respective secondary relevance scores. The re-rating results are shown in Table 2. Column 1 of Table 2 is the final relevant rating of the advertisement.

  The ad placement system 10 then selects one or more advertisements from the final match list 110 to relate to the web page 11 according to the rating of the final match list 110. In one embodiment, the selected advertisement is displayed with or linked to the web page 11.

  It will be appreciated that in other embodiments, the keyword related score may be designated as a primary related score and the TSV related score may be designated as a secondary related score. It is also understood that depending on the design preference, a different number of range levels may be used to transform the associated score. It is also understood that conditional or unconditional thresholds can be applied to related scores in any or all matched lists to shorten the list.

  In another embodiment, the system 10 may generate the final match list 100 by relying primarily on only one of the TSV match list 105 and the keyword match list 108. For example, the system 10 relies on a keyword matching list 108 that selects a preset number of advertisements according to their respective keyword-related scores. A TSV related score for each advertisement is also calculated. The advertisements on the keyword rating list 108 are then rerated based on their respective TSV related scores. The system 10 outputs the re-rated match list as the final match list 110.

  FIG. 3 illustrates another exemplary advertisement placement system 20 for associating one or more advertisements 12 with a web page 11 based on contextual association. For brevity of discussion, elements with the same reference number designation represent similar elements already discussed.

In the system 20, the TST and keyword semantic representation for the advertisement 12 is stored in the database 212. For each advertisement, database 212 provides two data fields: one for TSV and one for keyword semantic representation. The advertisement placement system 20 further includes a TSV and keyword indexer 211 for organizing and managing TSVs and keyword semantic expressions. The TSV and keyword indexer 211 can be implemented using a complete database management system (DBMS) or simply a software package for managing large data records, and the database 212 can be implemented with a database. Different indexing schemes can be applied to speed up the search.

  System 20 includes term extractors 102 and 112, TSV generators 103 and 113, and keyword selectors 106 and 115, all with the same functionality as previously described with respect to FIG. For each advertisement, TSV and keyword combiner 210 properly associates that TSV and keyword semantic representation with the advertisement. Similarly, for the web page 11, the TSV is generated by the TSV generator 103, and the keyword meaning expression is generated by the keyword selector 106. The TSV and keyword combiner 205 associates or links the TSV and keyword semantic expression with the web page 11. Information about TSV and keyword semantic representation for web page 11 and advertisement 12 is processed by TSV and keyword matcher 206, which includes TSV matcher 104 and keyword matcher 206 previously discussed with respect to FIG. Perform similar functions. Relevance scores for TSVs and keyword semantic expressions can be calculated as described with respect to FIG. The final match list 213 is generated by the TSV and keyword matcher 206 as previously discussed with respect to FIG.

  In another embodiment, a combined relevance score for each advertisement or each candidate or target data set may be calculated by combining keyword semantic representations and semantic vector representations of the data sets in the same vector space. For example, both the keyword representation and the semantic vector representation of the advertisement are treated as vectors in the same vector space and combined to form a single combined semantic vector representation of the advertisement.

  In calculating the combined vector semantic representation, different weights may be assigned to the semantic vector representation and the keyword semantic representation. For each advertisement, a relevance score is calculated based on the combined semantic vector representation of the advertisement and the combined semantic vector representation of the target data set. The final match list 213 is generated by the TSV and keyword matcher 206 according to each combined relevance score of the advertisement.

It will be understood that the match list generated based on keywords or TSV comparisons can be further refined or re-rated by other known methods. For example, a dataset or web page in a rating list may be stored in a “METHOD FOR NODE RANKING IN A LINKED”, the entire disclosure of which is incorporated herein by reference.
Using an algorithm according to link information between web pages in the final rating, such as the PageRank algorithm developed by Google, Inc., described in US Pat. No. 6,285,999, entitled “DATABASE” Can be rearranged.

Construction of TSV The structure of TSV for the data set will be described. Further details of TSV are described in US Pat. No. 6,751,621 and US application Ser. No. 11 / 126,184, the disclosures of which are already incorporated by reference.

  In preparation for generation of TSVs for a data set, a semantic dictionary is used to find TSVs corresponding to data points contained in the data set. The semantic dictionary includes known relationships between a plurality of known data points and a plurality of predetermined categories. In other words, the semantic dictionary contains “definitions”, ie TSVs of corresponding words or phrases.

An exemplary process for generating a TSV for a data set using a TSV generator will now be described. The data set can be an advertisement, a web page, or any type of data set. For illustration purposes, “word” is used as an example for a keyword contained in a document. Many other types of data points or keywords can be included in a document, such as words, phrases, symbols, terms, hyperlinks, metadata information, graphics and / or any displayed or hidden items or any combination thereof It is understood.

  Based on the input keyword of the document, the TSV generator identifies the corresponding keyword in the semantic dictionary and retrieves each TSV of each keyword contained in the document based on the definition given by the semantic dictionary. The TSV generator 103 generates a TSV of a document by combining each TSV of keywords included in the document. For example, the TSV of a document can be defined as a vector addition of each TSV of all keywords contained in the document.

  The process for creating a semantic dictionary will now be described. In one embodiment, a semantic dictionary is generated by appropriately determining which predetermined category each of a plurality of known data sets falls into. The sample data set can be in one or more predetermined categories, or the sample data set can be restricted to be associated with a single category. For example, a news report on a patent infringement lawsuit involving a computer company depends on the content of the report and on a predetermined category, such as "Intellectual Property Law" "Business Controversy" "Operating System" "Economic Issues" Can fall into a category containing If it is determined that the sample data set is associated with a certain predetermined category, all keywords included in the sample data set are associated with the same predetermined category. The same process is performed for all sample data sets.

  In one embodiment, the relationship between sample documents and categories can be determined by analyzing an Open Directory Project (ODP), which is enriched by hundreds of thousands of web pages by professional editors. Assigned to different topic hierarchies. These sample web pages with assigned categories are referred to as training documents for determining the relationship between keywords and predetermined categories. It should be apparent to those skilled in the art that other online topic hierarchies, classification schemes and ontologies can be used as well to relate sample training documents to categories.

  The following steps explain how the ODP hierarchy is transformed for the purpose of generating a TSV semantic dictionary.

  1. Download the ODP web page. An association is maintained between each web page and the ODP category to which it belongs. Remove any web pages that did not download properly and translate URLS to internal pathnames.

  2. Optionally, download all web pages referenced by any of the above ODP web pages and create an association between each new web page and the ODP category to which the original ODP web page belongs. Optionally filter the web page to keep only new web pages that have the same category as the original ODP web page from which it was pulled. Remove any web pages that did not download properly and translate the URL to an internal path name.

3. Optionally remove unwanted categories. Certain types of ODP categories are removed before processing. These removed categories identify empty categories (category with no corresponding document), letter bar categories ("movie titles starting with A, B ..." with no useful semantic differences), and semantic content Includes other categories that do not contain useful information (eg, empty categories, undesired foreign language regional pages), or misleading or inappropriate information (eg adult content pages) obtain.

  4). Remove pages that are not appropriate for training. In one embodiment, only pages with at least a minimal amount of content are used for training. In another embodiment, the training page must have at least 1000 bytes of converted text and up to 5000 whitespace delimited words.

  5. Optionally remove any pages that are not written in English. This can be done by standard methods such as HTML meta tags, automatic language detection, filtering on URL domain names, filtering on character ranges, or other techniques well known to those skilled in the art.

  6). Optionally remove duplicates. If a page appears in more than one ODP category, it is classified as ambiguous and may not be a good candidate for training.

  7). Identify candidate TSV dimensions. Invoke a collapse-trim algorithm to automatically flatten the ODP hierarchy and identify candidate TSV dimensions as described below.

  8). Optionally adjust the TSV dimension. Examine the automatically generated TSV dimensions and manually collapse, split or remove specific dimensions based on the expected semantic properties of those dimensions. The types of adjustments can include, but are not limited to: First, if certain words occur frequently in the original category name, those categories can be collapsed to their parent node (they all discuss the same thing or have semantic meaning) Because there is no one). Second, certain specific categories can collapse to their parents (usually because they are too specific). Third, certain groups of categories separated in the ODP hierarchy can be integrated (eg, “Arts / Magazines and E-Zines / E-Zines” can be integrated with “Arts / Online Writing / E-Zines”). Is).

  9. Create a TSV training file. For each potential training page, associate that page with the TSV dimension in which the page category collapsed. A page is then selected from each TSV dimension that will be used to train that dimension, taking care not to become overtrained or undersampled. In one embodiment, we randomly select 300 pages with converted text of at least 1000 bytes (if the page is less than 300, select them all). It then removes any page longer than 5000 blank separator words, starts with the smallest page, stops when the cumulative unit count reaches 200,000, and up to 200,000 blank separators for the entire dimension. Hold a word.

  10. Optionally relabel the dimensions. Each dimension starts with the same label as the ontology path of the ODP category from which it was derived. In one embodiment, some of the labels are manually adjusted to shorten and make them more readable and ensure that they reflect the various subcategories that have been combined or removed. For example, the original label “Top / Shopping / Vehicles / Motorcycles / Parts_and_Accessories / Harley_Davidson” may be rewritten to “Harley Davidson, Parts and Accessories”.

  In one embodiment, the collapse-cut algorithm proceeds from bottom to top in the ODP hierarchy, looking at the number of pages that are directly useful at each category node. If at least 100 pages are stored in the node, the node is retained as a TSV dimension. Otherwise, collapse it to the parent node.

  After the sample data set is assigned to a predetermined category (dimension), a data table that stores information representing the relationship between the keywords included in the one or more sample data sets and the predetermined category is displayed in the assignment result. Created based on. Each entry in the data table establishes a relationship between the keyword and one of the predetermined categories. For example, an entry in the data table can correspond to the number of sample data sets in the category that contain a particular keyword. Keywords correspond to the contents of the sample data set, while predetermined categories correspond to the dimensions of the semantic space. The data table may be used to generate a semantic dictionary that includes a “definition” of each word, phrase, or other keyword in a specific semantic space formed by a predetermined category that is used to construct a trainable semantic vector.

  FIG. 4 shows an exemplary data table for building a semantic dictionary. For simplicity and ease of understanding, the number of words and the predetermined number of categories in FIG. 4 are reduced to five. In practice, hundreds of thousands of terms and predetermined categories are possible.

As shown in FIG. 4, the table 200 includes a row 410 corresponding to predetermined categories Cat ₁ , Cat ₂ , Cat ₃ , Cat ₄ and Cat ₅ and representative words W ₁ , W ₂ , W ₃ , W ₄ and W. Column 412 corresponding to ₅ is included. Each entry 414 in table 200 corresponds to the number of documents having a particular word that occurs in the corresponding category, such as one or more words W ₁ , W ₂ , W ₃ , W _4, and W ₅ .

The sum of the total number of columns 412 across each row 410 gives the total number of documents that contain the word represented for that row 410. These values are represented in column 416. Referring to FIG. 4, the word W ₁ appears 20 times in the category Cat ₂ and 8 times in the category Cat ₅ . The word W ₁ does not appear in the categories Cat ₁ , Cat ₃ and Cat ₄ .

Referring to the column 416, the words W ₁ appears a total of 28 times across all categories. In other words, 28 classified documents contains W _1. Examining an exemplary column 412 such as Cat ₁ , word W ₂ appears once in category Cat ₁ , word W ₃ appears eight times in category Cat ₁ , and word W ₅ appears twice in category Cat ₁ . Word _{W 4} does not appear at all in the category Cat _1. As already mentioned, the word W ₁ does not appear in category 1. Referring to line 418, the entry corresponding to the category Cat ₁ indicates that the documents categorized Cat ₁ is 11.

According to one embodiment, after the data table is constructed, the importance of each entry in the data table is determined. The importance of an entry under certain circumstances can be considered as the relative strength that a word occurs in a particular category or its relevance to a particular category. However, this relationship should not be regarded as limiting. The importance of each entry is limited only to the actual data set and category (ie, features that are considered important to represent and describe the category). According to one embodiment of the present disclosure, the importance of each word is determined based on the statistical behavior of the word across all categories. This can be achieved by first calculating the percentage of keywords occurring in each category according to the following formula:
μ = Prob (entry | category) = (entry _n , category _m ) / category _{m_total}
Next, the probability distribution of the keyword occurrence over all categories is calculated according to the following formula.
ν = Prob (entry | category) = (entry, category _m ) / entry _{n_total}
Both μ and ν represent the strength with which a word is associated with a particular category. For example, if a word only occurs in a small number of datasets from a category and does not appear in any other category, it will have a high v value and a low μ value for that category. If an entry appears in most datasets from a category but also appears in some other category, it will have a high μ value and a low ν value for that category.

  Depending on the quality and type of information being represented, additional data manipulations can be performed to increase the determined importance of each word. For example, the value of μ for each category is normalized (ie, divided) by the sum of all values for a certain keyword, and as a result, interpretation as a probability distribution is possible.

A weighted average of μ and ν can also be used to determine the importance of keywords according to the following formula:
α (ν) + (1−α) (μ)
The variable α is a weighting factor that can be determined based on the information being represented and analyzed. According to one embodiment of the present disclosure, the weighting factor has a value of about 0.75. Other values can be selected depending on various factors such as the type and quality of information or the level of detail required to represent the information. Through empirical evidence collected from experiments, we have found that the weighted average of the μ and ν vectors is superior to the results achievable using only μ, using only ν, or using unweighted combinations of μ and ν. It was determined that a result could be produced.

FIG. 5 illustrates the operation processing described above based on the data from FIG. In FIG. 5, the table 230 stores a value indicating the relative strength of each word related to the category. Specifically, the ratio of keywords occurring in each category (that is, μ) is presented in the form of a vector for each word. The value for each entry in the μ vector is calculated according to the following formula:
μ = Prob (word | category) = (word _n , category _m ) / category _{m_total}
Table 230 also presents the probability distribution (ie, ν) of keywords across all categories in the form of vectors for each word. The value for each entry in the ν vector is calculated according to the following formula:
ν = Prob (category | entry) = (word _n , category _m ) / word _{n_total}
Referring to FIG. 6, a table 250 is shown to illustrate the semantic representation or “definition” of the words from FIG. Table 250 is a combination of five TSVs corresponding to the semantic representation of each word across the semantic space. For example, the first row, corresponding to the TSV of word _{W 1.} Each TSV has a dimension corresponding to a predetermined category. In addition, the TSVs for the words W ₁ , W ₂ , W ₃ , W ₄ and W ₅ are calculated according to the disclosed embodiment in which entries are scaled to optimize the importance of the word with respect to that particular category. More specifically, the value is calculated using the following formula:
α (ν) + (1−α) (μ)
An entry for each TSV is calculated based on the actual values stored in table 230. Accordingly, the TSVs shown in table 250 collectively constitute a semantic dictionary for a semantic space formed by predetermined categories, and the exemplary word W ₁ represented in FIG. 4 for each predetermined category or vector dimension. , W ₂ , W ₃ , W ₄ and W ₅ .

Sometimes it may be desirable to place advertisements in documents that are local to the market for the product being advertised. This can be accomplished by incorporating geographic information (zip code, city / state name, etc.) in the advertisement or by accessing the user's IP address and associating it with the geographic region. However, not all documents contain the proper form of geographic information, and not all users have IP addresses that correspond to their local regions. In this case, when the semantic dictionary as described above is formed, a further category related to the geographical area can be included in the predetermined category. Each geographic region becomes a dimension in the semantic space, and a sample data set tagged with geographic information is used to create a semantic dictionary. The semantic dictionary is then used to generate TSVs for the datasets and advertisements that reflect the strength with which the datasets and advertisements are associated with different geographic regions.

  The application of TSV is not limited to just one language. As long as a suitable sample data set is useful, it is possible to build a semantic dictionary for various languages. For example, an English sample data set from an Open Directory project can be replaced with an appropriate sample data set in another language when generating a semantic dictionary. There can be a separate semantic dictionary for each language. Alternatively, keywords for all languages can reside in a single common semantic dictionary. Different languages may share the same predetermined categories or semantic dimensions, depending on whether they share the same semantic dictionary and whether it is desired to compare semantic vectors across languages, or It can have completely different predefined categories or semantic dimensions.

After the semantic dictionary is created, the semantic dictionary can be accessed by the TSV generator 103 to find the corresponding TSV for the keyword included in the target document. In one embodiment, the keyword TSVs included in the target document are combined to generate a TSV for the target document. The way in which TSVs are combined depends on the specific implementation. For example, TSVs can be combined using vector addition operations. In this case, the TSV for the document can be expressed as:
TSV (document) = TSV (W1) + TSV (W2) + TSV (W3). . . + TSV (WN)
W1, W2, W3,. . . WN is a word included in the document.

  Generation of a TSV for a data set may utilize many types of information including keywords in the data set, information retrieved based on advertisements and keywords contained in the data set, and additional information assigned to the data set. For example, the words that appear in the advertisement, a set of keywords associated with each advertisement, the title of the advertisement, a brief description of the advertisement, market literature associated with the advertisement that describes the item being advertised, or it is sold Generation of TSVs for advertisements may be performed based on information from, but not limited to, viewers being viewed, and information from websites that may be referenced by the advertisements. TSV generation for a web page is the actual text that appears on the web page, or a metatext field associated with the web page, such as title, keyword and description, or linked to or linked by the web page. May be based on information including, but not limited to, some or all of text from other web pages.

  To speed up operation, TSVs for advertisements can be generated offline and can be updated as advertisements are modified, added or removed. However, TSVs can also optionally be generated at the time of advertisement placement. Similarly, TSVs for web pages or other data sets can be generated either off-line or on-the-fly.

According to an embodiment, the exemplary system disclosed herein parses various sections, such as web pages or displayed documents, and automatically assigns each section of one or more descriptions to a background item. Based on the final match list of Wikipedia (http:
// www. wikipedia. org) to a set of background items such as encyclopedic items.

  The methods and systems disclosed herein associate different portions of a data set that associate one or more advertisements with one or more web pages or documents, or vice versa, and retrieve related documents based on a user search query. Those skilled in the art will appreciate that the present invention is applicable to various purposes, such as finding background information. Also, a data set as used herein may be a single type of data set such as a web page or document, or a different type of data set such as a combination of email and web page, document and broadcast data. It is also understood that a collection of

  Another embodiment according to the present disclosure utilizes a precise representation referred to as a “tagged key” to represent or index data sets such as advertisements 12 and web pages 11. A tagged key associates keywords found in the dataset with one or more specific semantic categories applicable to the dataset. For example, the term “bank” can have many different meanings, but when tagged with a semantic category such as Financial Institution, it no longer matches a “bank” tagged with a semantic category such as Geologic Structure. .

  When analyzing a data set such as web page 11 or advertisement 12, candidate keywords that are considered to represent a web page or advertisement by keyword selector 115 or 106 as previously discussed for FIG. Alternatively, it is selected from the web page 11. In one embodiment, candidate keywords may be selected based on the frequency of each keyword appearing in a particular data set or document. An exemplary system according to the present disclosure accesses a semantic dictionary of information regarding the relationship between predetermined semantic categories and their candidate keywords. For example, for a data set having N candidate keywords and M predetermined categories, M × N pairs of keywords and categories (possible tagged keys) are valid. Filters can be used to remove categories that are not very relevant to keywords. A threshold that identifies the relevant minimum requirements may be used to identify categories that are sufficiently relevant to the keyword. One exemplary way of selecting a category for a keyword is simply to look up a semantic dictionary as discussed above, and this dictionary selects how powerful specific terms are for a given semantic category. Contains information that identifies In one embodiment, the category most strongly selected for the keyword will be the primary candidate for tagging.

  For example, assume that a document contains two keywords K1 and K2. At that time, one would look up K1 and K2 in the semantic dictionary to see which categories, if any, are connected to which keywords. If a keyword is associated with more than one category, such as categories C1, C2, C3 and C4, there are several options. That is, (1) select the category that has the strongest connection to the keyword, (2) select all categories that have connections that exceed the minimum threshold, or (3) select all categories regardless of the strength of the connection. . The result is a list of paired categories, keywords, and tagged keys such as K1 + C1, K2 + C2, and K2 + C4 to represent the data set. Each tagged key can be considered a semantic vector corresponding to the keyword, and the semantic vectors of the candidate keywords can be combined, such as by vector addition, to form a semantic vector representation of the data set. A semantic vector representation may be used as described in this disclosure.

FIG. 7 is a block diagram that illustrates a computer system 100 upon which an exemplary system of the present disclosure may be implemented. The computer system 100 includes a bus 702 or other communication mechanism for communicating information and a processor 7 coupled to the bus 702 for processing information.
04. Computer system 100 also includes main memory 706, such as random access memory (RAM) or other dynamic storage device, coupled to bus 702 for storing information and instructions executed by processor 704. Main memory 706 may also be used to store temporary variables or other intermediate information during execution of instructions executed by processor 704. Computer system 100 further includes a read only memory (ROM) 708 or other static storage device coupled to bus 702 for storing static information and instructions for processor 704. A storage device 710, such as a magnetic disk or optical disk, is provided and coupled to the bus 702 for storing information and instructions.

  Computer system 100 may be coupled via bus 702 to a display 712, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 714 that includes alphanumeric characters and other keys is coupled to the bus 702 to communicate information and command selections to the processor 704. Another type of user input device is a cursor control 716, such as a mouse, trackball or cursor direction key, for communicating direction information and instruction selections to the processor 704 and for controlling the movement of the cursor on the display 712. is there. This input device typically has two free axes in two axes, a first axis (eg, x) and a second axis (eg, y) that allows the device to specify the position of the plane. Have a degree.

  According to one embodiment of the present disclosure, the construction of TSVs and semantic operations includes one or more sequences of one or more instructions included in main memory 706 or storage device 710 or received from network link 120. Provided by computer system 100 in response to processor 704 executing. Such instructions may be read into main memory 706 from another computer readable medium such as storage device 710. Execution of the sequence of instructions contained in main memory 706 causes processor 704 to perform the processing steps described herein. One or more processors in a multi-processing arrangement may also be employed to execute the sequence of instructions contained in main memory 706. In alternative embodiments, the disclosure may be implemented using wired circuitry instead of or in conjunction with software instructions. Thus, the disclosed embodiments are not limited to any specific combination of hardware circuitry and software.

  The term “computer-readable medium” as used herein refers to any medium that participates in providing instructions to processor 704 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks, such as recording device 710. Volatile media includes dynamic memory, such as main memory 706. Transmission media includes coaxial cables, copper wire, and optical fibers, including the wires that make up bus 702. Common forms of computer readable media include, for example, floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, DVD, any other optical medium, punch card, Includes paper tape, any other physical media with a pattern of holes, RAM, PROM, and EPROM, FLASH-EPROM, any other memory chip or cartridge, or any other media that can be read by a computer .

Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to processor 704 for execution. For example, the instructions may initially be on a remote computer's magnetic disk. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 100 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infrared detector coupled to bus 702 can receive data carried in the infrared signal and place the data on bus 702. Bus 702 carries the data to main memory 706 from which processor 704 retrieves and executes instructions. The instructions received by main memory 706 may optionally be stored on storage device 710 either before or after execution by processor 704.

  Computer system 100 also includes a communication interface 718 coupled to bus 702. Communication interface 718 provides two-way data communication coupling to network link 120 connected to local network 722. For example, communication interface 718 may be an integrated services digital network (ISDN) card or modem and may provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 718 may be a local area network (LAN) card and provide a data communication connection to a compatible LAN. A radio link may also be implemented. In any such implementation, communication interface 718 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

  Network link 120 typically provides data communication to other data devices over one or more networks. For example, the network link 120 may provide a connection to a host computer 724 via a local network 722 or to a data device operated by an Internet service provider (ISP) 726. ISP 726 in turn provides data communication services through a global packet data communication network now commonly referred to as the “Internet” 728. Local network 722 and Internet 728 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals over the various networks that carry digital data to and from the computer system 100 and through the communication interface 718 on the network link 120 are exemplary forms of carrier waves that carry information.

  The computer system 100 can send messages and receive data including program codes via the network, the network link 120 and the communication interface 718. In the Internet example, server 130 may send a request code for an application program via Internet 728, ISP 726, local network 722 and communication interface 718. According to the disclosure, one such downloaded application provides TSV construction and execution of various semantic operations, as described herein. The received code may be executed by processor 704 as it is received and / or stored in storage device 710 or other non-volatile storage for later execution. In this way, the computer system 100 can obtain application code in the form of a carrier wave.

  In the preceding description, numerous specific details are set forth such as specific materials, structures, processes, etc., in order to provide a thorough understanding of the present disclosure. However, as will be appreciated by those having ordinary skill in the art, the present disclosure may be practiced without resorting to the details specifically set forth. In other instances, well known processing structures have not been described in detail in order not to unnecessarily obscure the present disclosure.

  Only the exemplary embodiments of the disclosure and their versatile examples are shown and described in this disclosure. It is to be understood that the present disclosure can be used in various other combinations and environments, and that changes or modifications within the scope of the inventive concept presented herein are possible.

Claims (25)

A machine-implemented method for controlling a data processing system for relating at least one data set from a group of data sets to a target data set, wherein each data set or the target data set has at least one keyword And the method is machine-implemented:
Accessing the semantic vector representing the target data set and a respective semantic vector representing each respective data set in the group;
Each semantic vector representing each respective data set in the group is between each of the at least one keyword in the respective data set and a predetermined category that can be related to each of the at least one keyword in the respective data set. Including collective information about the relationship
The semantic vector representing the target data set includes collective information on a relationship between the at least one keyword in the target data set and a predetermined category to which each of the at least one keyword in the target data set can relate. Including
The semantic vector representing each respective dataset in the target dataset or group has a dimension equal to the number of the predetermined categories;
For each data set in the group, by comparing the semantic vector associated with the target data set with the semantic vector associated with each data set in the group, each data in the target data set and the group Determining a first similarity between sets;
Accessing the keyword semantic representation of the target dataset and the keyword semantic representation of each respective dataset in the group;
The keyword semantic representation of the target data set or the keyword semantic representation of each respective data set in the group includes information representing a representative keyword of each data set in the target data set or the group;
The keyword semantic representation of the target data set or the keyword semantic representation of each respective data set in the group is different from the semantic vector of the target data set or the semantic vector of each respective data set in the group. Built in
For each data set in the group, by comparing the keyword semantic representation of the target data set with the keyword semantic representation of each data set in the group, between the target data set and each data set in the group Determining a second similarity of
The data set in the group according to the first similarity between the target data set and each data set in the group and the second similarity between the target data set and each data set in the group. Selecting at least one of:
Associating the at least one selected data set in the group with the target data set;
Including a method.   At least one of the data sets in the group is an advertisement, and the target data set is a document, web page, email, RSS news feed, data stream, broadcast data or information about a user, or one or more documents, web The method of claim 1, wherein the method is part of a page, email, RSS news feed, data stream, broadcast data or information about a user. The method of claim 1, wherein the target data set is part of a document, web page, email, RSS news feed, data stream, broadcast data, or user information.   The method further comprises: communicating to the user the at least one selected data set or a file associated with the selected data set with respect to the target data or a file associated with the target data set. The method described in 1.   The at least one selected data set displays the at least one selected data set, reproduces an acoustic signal according to the at least one selected data set, or the at least one selected data The method of claim 4, wherein the method is communicated to the user by providing a link to a set.   The at least one keyword is at least one of a word, a phrase, a character string, a pre-assigned keyword, a sub data set, meta information, and information extracted based on a link included in the respective data set. The method of claim 1 comprising:   The method of claim 1, wherein the semantic vector for each data set is pre-calculated and included in the respective data set.   The method of claim 1, wherein the semantic vector is dynamically generated. The semantic vector representing each respective data set in the group is a known vector between at least one keyword of each respective data set in the group and a predetermined category to which the known keyword can relate. Built on relationships,
The semantic vector representing the target data set is constructed based on the known relationship between at least one keyword of the target data set and a known keyword and a predetermined category to which the known keyword can relate. The method of claim 1.   The semantic vector associated with the respective data set is further generated based on information about at least one user or at least one data set linked to the respective data set. Method.   The method of claim 10, wherein the information about the at least one user includes at least one of previously viewed documents, previous search requests, user preferences, and personal information. According to a first similarity between the target data set and each data set in the group and a second similarity between the target data set and each data set in the group, at least of the data sets in the group The step of selecting one is
Designating one of the first similarity and the second similarity as primary similarity and the other as secondary similarity;
Accessing a plurality of preset related level information for the primary similarity;
For each data set in the group, mapping the primary similarity to one of the preset related levels according to the primary similarity;
Ranking the datasets in the group according to each mapped preset association level of the datasets in the group;
Within each association level, ranking the dataset to each association level according to the secondary similarity of the dataset;
Selecting the at least one of the data sets in the group according to a result of ranking the data set to each associated level;
The method of claim 1 comprising: The data in the group according to the first similarity between the target data set and each data set in the group and the second similarity between the target data set and each data set in the group. The step of selecting at least one of the sets is
Designating one of the first similarity and the second similarity as primary similarity and the other as secondary similarity;
Ranking the data sets in the group according to the primary similarity;
Selecting at least one candidate data set from the ranked data sets according to preset criteria;
Ranking the at least one candidate data set according to the secondary similarity;
Selecting the at least one of the data sets in the group according to a result of ranking the at least one candidate data set;
The method of claim 1 comprising: The data set in the group according to a first similarity between the target data set and each data set in the group and a second similarity between the target data set and each data set in the group The step of selecting at least one of
For each data set in the group, calculating a composite similarity according to a preset formula based on a respective first similarity of the data set and a respective second similarity of the data set;
Selecting the at least one of the data sets in the group according to a respective composite similarity of the data sets based on preset criteria;
The method of claim 1 comprising:   The method of claim 1, further comprising presenting the at least one of the data sets to a user simultaneously with the target data set.   The method of claim 1, further comprising presenting the at least one of the data sets to the user subsequent to presenting the target data set to the user.   The method of claim 1, wherein the at least one of the data sets or the target data set is presented to the user in an auditory form, a visual form, a video form, a tactile form, or any combination thereof. A data processing system for associating at least one data set from a group of data sets with a target data set, wherein each data set or the target data set includes at least one keyword, the system including data A data processor configured to process;
A data storage system configured to store instructions for controlling the data processor to perform the following steps when executed by the data processor;
Including
The step includes
Accessing the semantic vector representing the target data set and a respective semantic vector representing each respective data set in the group;
Each semantic vector representing each respective data set in the group is between each of the at least one keyword in the respective data set and a predetermined category to which each of the at least one keyword in the respective data set can relate. Including collective information about the relationship
The semantic vector representing the target data set includes collective information of a relationship between the at least one keyword in the target data set and a predetermined category to which each of the at least one keyword in the target data set can be related. ,
The semantic vector representing each respective dataset in the target dataset or group has a dimension equal to the number of the predetermined categories;
For each data set in the group, by comparing the semantic vector associated with the target data set with the semantic vector associated with each data set in the group, each data in the target data set and the group Determining a first similarity between sets;
Accessing the keyword semantic representation of the target dataset and the keyword semantic representation of each respective dataset in the group;
The keyword semantic representation of the target data set or the keyword semantic representation of each respective data set in the group includes information representing a representative keyword of the respective data set in the target data set or the group,
The keyword semantic representation of the target data set or the keyword semantic representation of each respective data set in the group is different from the semantic vector of the target data set or the semantic vector of each respective data set in the group. Built in
For each data set in the group, by comparing the keyword semantic representation of the target data set with the keyword semantic representation of each data set in the group, between the target data set and each data set in the group Determining a second similarity of
The data set in the group according to the first similarity between the target data set and each data set in the group and the second similarity between the target data set and each data set in the group. Selecting at least one of:
Associating the at least one selected data set in the group with the target data set;
Including the system. A machine-readable medium carrying instructions for controlling a data processing system to perform machine-executed steps and relate at least one data set from a group of data sets to a target data set during execution of the data processing system. Each data set or the target data set includes at least one keyword, and the step includes:
Accessing the semantic vector representing the target data set and a respective semantic vector representing each respective data set in the group;
Each semantic vector representing each respective data set in the group is between each of the at least one keyword in the respective data set and a predetermined category to which each of the at least one keyword in the respective data set can relate. Including collective information about the relationship
The semantic vector representing the target data set includes collective information on a relationship between each of the at least one keyword in the target data set and a predetermined category to which each of the at least one keyword in the target data set can relate. Including
The semantic vector representing each respective dataset in the target dataset or group has a dimension equal to the number of the predetermined categories;
For each data set in the group, by comparing the semantic vector associated with the target data set with the semantic vector associated with each data set in the group, each data in the target data set and the group Determining a first similarity between sets;
Accessing the keyword semantic representation of the target dataset and the keyword semantic representation of each respective dataset in the group;
The keyword semantic representation of the target data set or the keyword semantic representation of each respective data set in the group includes information representing a representative keyword of each data set in the target data set or the group;
The keyword semantic representation of the target data set or the keyword semantic representation of each respective data set in the group is different from the semantic vector of the target data set or the semantic vector of each respective data set in the group. Built in
For each data set in the group, by comparing the keyword semantic representation of the target data set with the keyword semantic representation of each data set in the group, between the target data set and each data set in the group Determining a second similarity of
The data set in the group according to the first similarity between the target data set and each data set in the group and the second similarity between the target data set and each data set in the group. Selecting at least one of:
Associating the at least one selected data set in the group with the target data set;
Including medium. A machine-implemented method for controlling a data processing system for associating at least one data set from a group of data sets with a target data set, wherein each data set or the target data set is at least 1 Including two keywords, the machine-executed step comprises:
Accessing the semantic vector representing the target data set and a respective semantic vector representing each respective data set in the group;
Each semantic vector representing each respective data set in the group is between each of the at least one keyword in the respective data set and a predetermined category to which each of the at least one keyword in the respective data set can relate. Including collective information about the relationship
The semantic vector representing the target data set includes collective information on a relationship between each of the at least one keyword in the target data set and a predetermined category to which each of the at least one keyword in the target data set can relate. Including
The semantic vector representing each respective dataset in the target dataset or group has a dimension equal to the number of the predetermined categories;
Accessing the keyword semantic representation of the target dataset and the keyword semantic representation of each respective dataset in the group;
The keyword semantic representation of the target data set or the keyword semantic representation of each respective data set in the group includes information representing a representative keyword of the respective data set in the target data set or the group;
The keyword semantic representation of the target data set or the keyword semantic representation of each respective data set in the group is different from the semantic vector of the target data set or the semantic vector of each respective data set in the group. Built in
Generating, for each data set, a combined vector representation of the data set according to the semantic vector associated with each data set and the keyword semantic representation of each data set;
Generating, for the target data set, a combined vector representation of the target data set according to the semantic vector associated with the target data set and the keyword semantic representation of the target data set;
Determining similarity between the target data set and each data set in the group by comparing the combined vector representation of the target data set and the combined vector representation of each data set in the group; ,
Selecting at least one of the data sets in the group according to the determined similarity;
Associating the at least one selected data set in the group with the target data set;
Including a method. A machine-readable medium carrying instructions for controlling a data processing system to perform machine-executed steps and relate at least one data set from a group of data sets to a target data set during execution of the data processing system. Each data set or the target data set includes at least one keyword, and the step includes:
Accessing the semantic vector representing the target data set and a respective semantic vector representing each respective data set in the group;
Each semantic vector representing each respective data set in the group is between each of the at least one keyword in the respective data set and a predetermined category to which each of the at least one keyword in the respective data set can relate. Including collective information about the relationship
The semantic vector representing the target data set includes collective information on a relationship between each of the at least one keyword in the target data set and a predetermined category to which each of the at least one keyword in the target data set can relate. Including
The semantic vector representing each respective dataset in the target dataset or group has a dimension equal to the number of the predetermined categories;
Accessing the keyword semantic representation of the target dataset and the keyword semantic representation of each respective dataset in the group;
The keyword semantic representation of the target data set or the keyword semantic representation of each respective data set in the group includes information representing a representative keyword of each data set in the target data set or the group;
The keyword semantic representation of the target data set or the keyword semantic representation of each respective data set in the group is different from the semantic vector of the target data set or the semantic vector of each respective data set in the group. Built in
Generating, for each data set, a combined vector representation of the data set according to the semantic vector associated with each data set and the keyword semantic representation of each data set;
Generating, for the target data set, a combined vector representation of the target data set according to the semantic vector associated with the target data set and the keyword semantic representation of the target data set;
Determining similarity between the target data set and each data set in the group by comparing the combined vector representation of the target data set and the combined vector representation of each data set in the group; ,
Selecting at least one of the data sets in the group according to the determined similarity;
Associating the at least one selected data set in the group with the target data set;
Including medium. A machine-implemented method for controlling a data processing system for relating at least one data set from a group of data sets to a target data set, wherein each data set or the target data set has at least one keyword And the machine-executed steps include:
Accessing a tagged key representation representing the target data set and a respective tagged key representation representing each respective data set in the group;
Each tagged key representation representing each respective data set in the group has a relationship between each representative keyword of each respective data set and a predetermined category to which each said representative keyword in each respective data set can be related. Including collective information,
The tagged key representation representing the target data set includes collective information of a relationship between each representative keyword in the target data set and a predetermined category to which each representative keyword in the target data set may be related,
For each data set in the group, comparing the tagged key representation associated with the target data set with the tagged key representation associated with each data set in the group, and Determining the similarity between each dataset in the group;
Selecting at least one of the data sets in the group according to the determined similarity between the target data set and each data set in the group;
Associating the at least one selected data set in the group with the target data set;
Including a method. When executing a data processing system, machine-readable, carrying instructions that control the data processing system to perform machine-executed steps to relate at least one data set from a group of data sets to a target data set A medium, wherein each data set or the target data set includes at least one keyword, the step comprising:
Accessing a tagged key representation representing the target data set and a respective tagged key representation representing each respective data set in the group;
Each tagged key representation representing each respective data set in the group has a relationship between each representative keyword of each respective data set and a predetermined category to which each said representative keyword in each respective data set can be related. Including collective information,
The tagged key representation representing the target data set includes collective information of a relationship between each representative keyword in the target data set and a predetermined category to which each representative keyword in the target data set may be related,
For each data set in the group, comparing the tagged key representation associated with the target data set with the tagged key representation associated with each data set in the group, and Determining the similarity between each dataset in the group;
Selecting at least one of the data sets in the group according to the determined similarity between the target data set and each data set in the group;
Associating the at least one selected data set in the group with the target data set;
Including medium. A machine-implemented method for controlling a data processing system to generate a tagged representation of a data set that includes at least one keyword, the method comprising:
Identifying a representative keyword from the at least one keyword to represent the data set;
Accessing data identifying a known relationship between each known keyword and a given category;
Determining a relationship between each representative keyword and the predetermined category by referring to the accessed data;
Building a tagged key representation of the data set according to the determined relationship between each representative keyword and the predetermined category;
Representing the data set using the constructed tagged key representation;
Including a method. When executing a data processing system, machine-readable, carrying instructions that control the data processing system to perform machine-executed steps to relate at least one data set from a group of data sets to a target data set A medium, wherein each data set or the target data set includes at least one keyword, the step comprising:
Identifying a representative keyword from the at least one keyword to represent the data set;
Accessing data identifying a known relationship between each known keyword and a given category;
Determining a relationship between each representative keyword and the predetermined category by referring to the accessed data;
Building a tagged key representation of the data set according to the determined relationship between each representative keyword and the predetermined category;
Representing the data set using the constructed tagged key representation;
Including medium.

Images