KR20120005012A - Domain status, purpose and categories - Google Patents

Abstract

Systems and methods of creating and using a domain profile include checking the status of the first page associated with a domain. The first page is retrieved, and additional pages from the domain are identified based on hyperlinks from the first page. The status of the additional page is identified and the hyperlinks are prioritized based on the status and / or comparison with predetermined data. The content is extracted from the page selected among the first page and the additional page. Certain additional pages may be selected based on their priority. The retrieved content may be processed through a set of signature markers to determine contextual matches. The purpose of the domain is determined by the state of the first page, the state of the additional page and the results of the processing of the content. The domain profile can be displayed, stored, sent and / or searched to identify web sites or attributes of interest.

Description

Domain status, purpose and category {DOMAIN STATUS, PURPOSE AND CATEGORIES}

As the number of web sites operating on the Internet increases, the demand for organizing and evaluating available content is increasing. Search engines have made it easier for individual users to find and access available information. Typical search engines may include algorithms that help identify relevant content based on a user's search terms. These may include, for example, considering the apparent popularity of a particular page with the requested content based on web traffic. However, there are typically limitations on the capabilities of conventional services to download content from a web site, index it, and then allow users to browse based on the downloaded content.

Understanding how the Internet is inherently organized can help you understand the problems associated with the efficient use of the full range of information that can be accessed through the Internet. Domain Name System. (DNS) is part of the Internet infrastructure that translates human-readable domain names into Internet Protocol (IP) numbers needed to establish TCP / IP communications over the Internet. That is, DNS uses Web sites, and user-friendly domain names such as "www.example.com" rather than numerical IP addresses such as "123.4.56.78" assigned to computers on the Internet. Allows you to reference other resources. Each domain name consists of a series of character strings (labels) separated by dots. The rightmost label in the domain name is known as the "top-level domain" (TLD). Examples of known TLDs are ".com"; ".net"; ".org." And so on. Each TLD supports second-level domains, listed just to the left of the TLD, such as the "example" level at "www.example.com". Each second-level domain may include a number of third level domains located just to the left of the second-level domain, such as, for example, the "www" level at "www.example.com". There may be other level domains with virtually no restrictions. For example, a domain with additional domain levels may be "www.photos.example.com".

It is also noted that a single IP address, for example a single server, can support numerous domain names. In other words, different domain names may be resolved to the same server, which may determine what content to provide based on the requested domain name and / or additional non-domain information. This is sometimes called virtual hosting.

Additional non-domain information may be included in a Uniform Resource Identifier (URI) structure that includes a domain name. For example, the "path" part is a series of segments (not necessarily representing directories, but conceptually similar), separated by diagonal lines ("/"). This information may be included just to the right of the domain name, such as "blog" at "www.example.com/blog" and may be used by a server or other receiving device to identify and deliver specific content or to execute specific code. Can be. Other examples of non-domain information may include queries and fragments, the details of which are known to those skilled in the art and are not discussed in detail herein. Combinations of this information may be included in web page hyperlinks that navigate the user to another part of the same page or to another web page that may be part of the same or different domains.

Relevant domain names, and content are "www.example.com"; "www.blog.example.com"; "www.example.com/blog"; Or "blog.example.com" and the like, each of which may be configured in a hierarchical or nested manner with different importance. These related domains do not need to share similarities in the actual IP address that resolves the various domain names. In this regard, part of the domain name may mean a particular server as desired, for example, "mail.example.com" and www.example.com "may be used to describe different functions for the same second-level domain. Can resolve to different servers.

The task of operating each TLD (including maintaining a registry of second-level domains in the TLD) is delegated to a particular organization known as a domain name registry ("registry"). The registry is primarily responsible for translating domain names into IP addresses ("resolving") and running its top-level domains via DNS servers that maintain this information in large databases.

The present invention can provide advantages in improving accessibility and meaning of content related to domains. By taking into account domain features that were not systematically recognized, users can search for properties of a domain and, in a wider application, use a domain's web site using information publicly available through DNS servers and web sites. A profile may be built for. As used herein, domain features include data content that is part of a web page, data content that is linked to a web page, data content that is part of a web page's DNS resolution support architecture, and the like, but these It is not limited to. Thus, given a domain name, associated information such as attributes and features of the web site may be provided based on an expanded search based on the domain name. The associated information may provide improved search results based on the attributes describing the domain and / or the content of the domain. From these profiles, improved analysis of some available content on the Internet can be done that can be used to improve the efficiency and productivity of individual users, business activities, and generally web content support services.

Embodiments of the invention relate to systems and methods for analyzing a domain. Embodiments may include checking the status of the first page associated with the domain. Checking the status may include attempting to resolve the domain name of the first page, and if it is unable to resolve the domain name, check the status of the domain as non-functional. If the domain name resolves but the content is inaccessible, the status can be verified as inaccessible.

If the first page can be accessed, the content of the first page can be retrieved. Multiple additional pages of the domain may be identified based on hyperlinks from the first page. The state of the additional page can be checked as it was in the first page. That is, checking the status of additional pages may include attempting to resolve the domain name or network address of the first additional page, and if the domain name or network address cannot be resolved, the status of the first additional page is It can be identified as non-functional. If the domain name is not resolved or the content is inaccessible, then the status of the first additional page may be identified as inaccessible, and so on for other pages.

In addition, hyperlinks may be prioritized based on status and / or comparison with predetermined data. For example, the strings in a URI may be determined in advance as having a particularly high or low importance. Thus, hyperlinks containing such strings may be prioritized accordingly. The content may be extracted from selected pages of the first page and a plurality of additional pages. The content may include the full range of domain features described herein. The selection of specific pages among the additional pages may be based on their priority.

The content of the first page and the additional page may be processed through a set of predetermined or generated data to determine contextual matches between the content. The predetermined or generated data will be referred to as signature marker set. The signature marker set should be understood as data in one table that correlates known elements to some other subject. For example, known terms may be associated with the frequency with which terms appear on a sampling of domain names. The comparison of terms found on the page to the domain frequency of terms may be the first step in processing the term through the signature marker set. In embodiments, this allows for a more accurate determination of domain profile by focusing on terms with low domain frequency, which may be more distinct than terms with high domain frequency. In addition, the signature marker set may use techniques to reduce potential term ambiguity. For example, the content may be compared with predetermined data representing known associations of multiple data content. This may include known text associations, known data-type associations, and various combinations of similar data contained within domain features. Therefore, the importance of the first subject can be determined more accurately by the identification of the second associated subject in the domain features. As mentioned above, domain features may include data from or linked to the web page itself, or DNS information such as an IP address or URL.

Based on contextual matches, the state of the first page, and the state of the additional page, the purpose of the domain can be determined. Embodiments may include executable code whose content is nested within a first page or additional page, and a set of signature markers configured to determine contextual matches within the executable code. Also, embodiments may include image, video, and audio information whose content is embedded in a first page or additional page, and wherein a set of signature markers is configured to determine a contextual match within the image, video, and audio information. Can be.

Once determined, the domain purpose can be used in a number of different ways, including searching, displaying, storing and / or sending the determined purpose of the domain. Embodiments may include identifying and retrieving related domains based on the determined state and purpose of the domain.

Embodiments may include determining the purpose of the domain in the content based on at least one of a determined domain name, a registered sales marker, the absence of a particular data type, an alternative publisher of the page, a community identifier, and a data type. have.

In addition, embodiments may include identifying these based on the purposes of one or more domains. The category of identified domains can be determined to be distinct from the respective purposes of the domains. Related domains can be identified and searched based on the domain's determined purpose and categories.

Embodiments may include following hyperlinks from the first page by traversing redirects to the https version of the web page.

Embodiments include receiving a set of purposes input from a user; Identifying attributes common to input domains among contextually matched content; And outputting the verified domains to the user.

Embodiments include receiving a set of purposes and / or categories input from a user; Identifying domains corresponding to the entered set of purposes and / or categories; And outputting the verified domains to the user.

Embodiments may include iteratively performing the methods described for a range of domains and compiling the results of a historical analysis of the domain domain involved.

Embodiments of the present invention may include systems implementing the described methods as well as computer-readable storage media encoded with instructions for causing a computer to execute the described methods. For example, an electronic system including a processor, a memory, and an electronic communication device may, via the electronic communication device, identify the status of the first page associated with the domain and retrieve the first page; Identify a plurality of additional pages from the domain based on the hyperlinks from the first page; Confirm the status of the additional page via the electronic communication device; Prioritize the hyperlinks based on a comparison with the predetermined data; Extract content from at least one of the first page and the plurality of additional pages via the electronic communication device, wherein at least one page of the plurality of additional pages is selected based on priority; Process the content through the signature marker set to determine a contextual match; Determine the purpose of the domain according to the status of the first page, the status of the additional page, and the processing results of the content; And configured to perform at least one of those that display, store, and send the determined purpose of the domain.

Other advantages of the present invention will become apparent to those skilled in the art upon reading and understanding the following detailed description of the preferred embodiments.

1 illustrates an exemplary system according to an embodiment of the invention.
2 illustrates an exemplary method according to an embodiment of the invention.
3 illustrates an exemplary web page, and associated content, in accordance with an embodiment of the invention.
4 graphically illustrates an exemplary layered data resolution, in accordance with an embodiment of the invention.
5 graphically illustrates an example process flow in accordance with an embodiment of the invention.
6 illustrates aspects of an exemplary method according to an embodiment of the invention.

Embodiments of the present invention may help a user, web content provider, and / or registry understand how to target domain names. A holistic view of online activity by industry sector can be built to improve understanding of the online environment and to complement other microanalytic tools. Thus, the systems and methods described herein may produce results that can provide domain site-purpose data.

Given a domain name, the described systems and methods can derive attributes / characteristics about a web site that can be accessed via the domain name. In addition, the service allows users to search domains based on criteria that may include a domain or some attributes describing content on the domain. Embodiments may use information contained in publicly available DNS servers and content available in publicly available websites to build a profile about the domain's website.

Embodiments may collect and provide different types of data associated with a domain in a hierarchical form, for example, as shown in FIG. 1. Embodiments may determine domain status by attempting to load a web page, determining domain purpose and domain categories by gathering text or other data from the website, and passing it through a set of signature markers.

The systems and methods include example attributes regarding websites of a domain, including domain status; Domain purpose; Domain categories; Attempts to gather information about domain traffic, domain key terms, domain characteristics / features / functions, and domain content. These attributes are further described below. Through the described data collection and analysis, embodiments of the present invention can provide an overview of their use for improved directory and website hosting of TLDs, such as .com and .net TLDs. For example, by determining domain profiles for all or some domains within a TLD, related domains may be identified according to patterns that exist across domain-space-wide rather than categories determined and applied to individual web page content. An improved directory for sorting can be created. This can provide advantages not previously recognized in managing the various levels of Internet architecture and services. For example, by determining the status, purpose, and categories of domains rather than individual web pages alone, individual users, content providers, and registries may be better aware of the relevance of the content and the marketing and other aspects of the various uses of the Internet. Be able to better recognize patterns directly related to important aspects.

The following example method is described with reference to FIGS. 2 and 3. As shown in FIG. 2, the method may begin with determining a state of a first page (S1000). Domain status generally relates to whether and how to resolve a domain. For example, it may be determined whether there is a web server associated with the domain, and if so, whether the web server can connect. Another information may include whether there are any specifically identified web server errors. For example, a domain name is entered and sent to a DNS server to attempt to resolve the domain name. If the domain name cannot be resolved, the state of the domain may be determined to be non-functional. If the domain name resolves but the content is inaccessible, the status may be identified as inaccessible. Other state identifiers are possible.

Domain status may indicate the operational status of an associated website, eg, active, HTTP errors, and so forth. The domain status can indicate whether a website can be accessed from a particular domain, and if not, at what stage did not attempt to access this website. Access failures may include error codes assigned to domain web sites such as those shown in Table 1.

Domain status Status title State Description it does not exist No name servers found. Name Server Error The name server cannot translate domain requests to Internet (IP) addresses. Name Server Timeout The name server does not respond within the specified time period or after the specified number of attempts. HTTP Error 4XX Web servers respond with content errors. HTTP Error 5XX Web servers respond with a server error No robots allowed The server refuses to access the crawler. Host not reachable Web servers do not respond Unknown web error The servers respond with an unknown error message. OK The server can reach the content and respond.

If one or more queries in the domain are found to be non-functional or otherwise inaccessible errors, the method proceeds from step S1010 to step S1700, where the purpose of the domain is determined based on the determined state. For example, a status code, such as a name server error, may be used to determine a general non-functional purpose related to an error code, or a more specific non-functional purpose.

If the domain name resolves but results in a redirect, it can be included in determining the state of the domain. For example, the status can be determined to be a "redirect" domain, with no active content.

If the web server for the domain name does not result in a successful connection and redirect, the content of the first page 500 of the example shown in FIG. 3 may be retrieved in step S1100.

In step S1100, content, for example elements 502, 504, 506, 508, are retrieved from the first page. This content may be in various forms known to those skilled in the art, including, for example, text, multimedia, hyperlinks, or other executable code. By way of example, elements 502, 504, 506 may be web page buttons that activate hyperlinks to web page 510, 520, 530, respectively. Element 508 may be text, image, or other multimedia data content. The content may be used for at least two purposes described herein. One purpose is any hyperlinks associated with the first page, for example as hyperlinks embedded in the first page, for example, based on elements 502, 504, 506 that can be accessed through the first page. It may be to check another page. This function may be performed in step S1200. That is, a plurality of additional pages, for example, 510, 520, and 530, may be identified based on the hyperlinks detected in the first page. Although the illustrated pages 510, 520, 530 share the same second level domain, other pages associated with, for example, hyperlinked from, page 500 do not necessarily share the same domain. Another purpose of the content may be to help determine the purpose of the domain, which is further described below. The method continues with step S1300.

In operation S1300, additional pages, for example, states of 510, 520, and 530 may be checked. Checking the status of additional pages may include attempting to resolve the domain name or network address of the first additional page, and if the domain name or network address cannot resolve, the status of the first additional page may be It can be identified as non-functional. If the domain name resolves but cannot be accessed, the state of the first additional page may be identified as functional but inaccessible. As discussed above with regard to determining the state of the first page 500, other state identifiers are possible. The method continues with step S1400.

In step S1400, the identified hyperlinks and associated additional web pages may be prioritized based on comparison with predetermined data. For example, hyperlink data, including domain, and non-domain URI information, can be compared to predetermined markers, such as lists of significant keywords, strings or other values that present the desired content. . The structure of the hyperlink data can be analyzed as part of the process for recognizing patterns of information, eg, particular nested forms and the like. As a result, a prioritized list of identified hyperlinks may be generated.

The user can select specific markers to use to customize the priority. This may include presenting to the user a list of markers that have been recognized in the identified hyperlink data and allowing the user to select among these markers. To further assist the user, the user may be provided with optional additional information related to the recognized markers. For example, each of the recognized markers may be given a quantitative or qualitative value to assist the user in evaluating which markers have the greatest importance to the user or related domain. For example, a user may be particularly interested in certain identified markers, or a set of markers may appear to be more relevant based on the quantitative value of the presence of a marker in the hyperlink data. Thus, the user can select from the markers based on additional predetermined information related to the particular markers presented to the user. This can provide advantages in identifying more relevant web pages from which to extract content as part of domain profiling. The method continues with step S1500.

In step S1500, additional content may be extracted from the first page 500, such as content 508, if necessary, and the content may be extracted from a selected page of the plurality of additional pages. The selection of specific pages among the additional pages may be based on their priorities. For example, if page 530 is prioritized to additional pages 510, 520, 530 at the lowest priority, content may only be collected from pages 510, 520. In embodiments, page 530 may be designed as a low priority based on the text string "mail" in the domain name. This can save resources and lead to more accurate results in later steps by identifying the most relevant additional pages and content. The type of collected content, discussed further below, may be determined based on the parameters of the signature marker set. For example, the text content may be all the data needed if the signature marker set is configured for text only. This determination may provide efficiency in large data collection and analysis. If the signature marker set is configured to handle multiple data types, this may improve the overall accuracy of the analysis.

Following the hyperlink data from the first page may include traversing the redirect. For example, the hyperlink data may not resolve directly to another web page, but may instead require at least one redirect. Thus, additional pages may be indirectly linked or associated with the first page. This may include traversing the redirect to the https version of the web page. The method continues with step S1600.

In step S1600, content collected from the first page and the additional page may be processed through a set of signature markers to determine contextual matches between the content. As described above, the signature marker set may include linked data elements that provide contextual match, or importance, to one of the data elements. By identifying the contextual matches of the collected content, weights can be given that are appropriate for the importance of the various parts of the content. For example, the term "Ford" is a car manufacturer, but this is also the last name of a common name, which alone has an uncertain importance. This may result in improperly considering or ignoring this term as part of web page content. Eliminating the term ambiguity may require that "Ford" and other identified automotive terms be in close proximity to each other in order for the use of the term "Ford" to be considered relevant to the classification of the vehicle. N-gram models are a type of probabilistic model that predicts the next item in a sequence. N-grams are used in various fields of statistical natural language processing and genetic sequence analysis and can be applied to the present invention to refine the content processing described herein. For example, the n-gram model predicts xi based on xi-1, xi-2, ..., xi-n. As described above with respect to markers used to prioritize hyperlink data and additional pages, the method allows the user to select the desired association among the identified associations. For example, a user may identify certain identified contextual matches as valid and other contextual matches as invalid or to be ignored. In addition, the methods may assist the user in identifying the importance of contextual matches by providing additional quantitative or qualitative information regarding contextual matches. Thus, automated methods can be used to evaluate the weight given to contextual matches, as in the case of predetermined signature marker sets that are automatically applied to the extracted content, or in doing so, It can help the user, such as in cases where the user can accept, weight, or reject contextual matches.

Regarding the signature marker set, the particular types of data processed from the content are not limited to text or other specific data. For example, embodiments may include a set of signature markers such that the content includes executable code embedded in a first page or additional page and determines a contextual match within the executable code or within the results of the executable code. It may include being configured. In addition, embodiments include image, video and / or audio information whose content is embedded in a first page or additional page, and a set of signature markers may be set to determine contextual matching within any of the image, video and audio information. It may include being configured. For example, there may be contextual matches between various audio data, such as pattern recognition results, audio type, audio length, or any of a number of related attributes. A large number of similarly patterned audio information can be, for example, certain indicators of a particular type of web page that specifies the content of a particular music artist, and can therefore have particular importance.

In the case of both hyperlink markers and signature marker sets, marker performance reporting may be included to provide improved performance. For example, a user may be provided with a qualitative assessment of the performance of individual markers. Alternatively, or in connection with automated assessment, a user can independently evaluate and rank the effectiveness of the markers. Therefore, the system can ensure that the markers used are valid by providing a report on marker performance and allowing it to be varied in use of the markers. The report can be provided to the user so that the user can implement the changes, or the system can be automated to discard poorly performing markers, such as markers that fall below some threshold. After processing the content, the method continues at step S1700.

In step S1700, the purpose of the domain may be determined based on the state of the first page, the states of any additional page, and any results of the contextual match. Domain purposes may relate to the intended use or uses of subject or overall meaning or content associated with a domain. The purpose may reflect the obvious reason why the domain is used. For domains that cannot resolve or send other access errors, the purpose may be that the domain is not in use or restricted. Other identified objectives may include Pay Per Click (PPC), parking, purpose, masked redirection, redirection, blogging, and the like, as detailed in Table 2 below.

The results of contextual matches can be particularly effective in determining the specialized purposes of the active domains. The accuracy of this assessment is improved by further analysis of prioritized additional pages. Thus, functional web sites may be assigned non-exclusive object codes, such as those identified in Table 2, and any other suitable code for specifying other purposes.

purpose title Explanation block The domain keeps robots from crawling. PPC Purpose Main name registered (by the operator or registrar of the domain) only to drive advertising traffic PPC Park Existing domain names not set up by end user registrants and registrars have created parked names for other purposes to drive revenue traffic. Park The domain name uses a conventional registered park without advertising links. In production Domain name is being produced without advertising links. Redirect The domain redirects to another domain. Masked
Redirect Display only content from sites with different domains. adult The domain displays adult content. Company / Institution Domain names are used to provide information about a company or organization. blog Domain names are mainly used for blogs. news The domain gathers and displays news. E-commerce Domains are used to sell products online. forum Domains are primarily used by users to discuss various related topics. Social networking Domains are often used to make it easier for individuals with similar ideas to get along.

Additional information may be included in the purpose decision. For example, measurements of domain traffic, such as statistics primarily from DNS traffic processors, may be included to evaluate whether a web site is a real retail site. Domain key terms, such as title, title and description, may be given specific weights in addition to the contextual match weights described above. Domain characteristics, features, and / or functions indicate whether common features are on a website, and additional pages may be used in determining purposes such as blogs, retail, and the like.

In addition, in determining the purpose, it may be advantageous to consider the technical details of the content. For example, identifying what types of technologies websites use, for example, mail servers, cookies, multimedia, SSL certificates for securing data, etc., associated with a domain may be used for purposes such as retail-to-person, Or other indications may be provided. Additional technical data may be displayed other than the specific content, such as the average time of displaying the content on the associated page, the geographic location of the web server, and the like.

Embodiments may include determining the purpose of a domain in the content based on at least one of a determined domain name, a registered sales marker, the absence of a specified data type, an alternative publisher of the page, a community identifier, and a data type. have. That is, the presence or absence of specific content, as listed above, may have independent importance in determining the overall purpose of the domain. Examples may include indications of advertiser information, or absence thereof, or data types such as image data, or absence thereof. Once the domain purpose is determined, the method continues at step S1800.

In step S1800, a category of a domain different from the respective purposes of the domain may be determined. Domain categories may reflect categories associated with the business units related to the content on the site. This may include determining which categories the content belongs to from the first page and the additional page. For example, domain categories can conform to the North American industry classification system to put domains into business classification systems. Table 3 is a partial list of example categories that may include an assigned prioritization of categories within a category code.

Category code General category Terminology in Category Segments One Restrictions 1A Pharmacies 1B Drugs / Drugs 1C pharmacy 2 fitness 2A Exercise yoga 2B Diet and Nutrition Diet meal 2C volunteer 2D Review site

The information used to determine the domain category may include content obtained from the first page and additional pages, or may even be the same content used to determine domain purpose. However, the importance attributed to the various information may be different in each process. For example, as discussed above, the presence of "Ford" along with other contextually relevant company information may be used to determine the COMPANY / ORGANIZATION purpose of the web page. The presence of the same information "Ford" along with other car information may be used to determine domain categories in car technologies.

As in the determination of domain purpose, domain classification is not limited to the content of a particular web page. Rather, the domain classification may be derived from the content of the first page and additional pages prioritized in order to reach a comprehensive classification of the domain. Embodiments may include identifying these domains based on the purposes of one or more domains prior to performing classification of the domains. This can provide advantages in accurately categorizing these domains into classifications, with or without user interaction.

As discussed embodiments of the present invention, features of domain purpose and domain categories may differ. That is, the domain purpose may include the purpose of a website associated with the domain, such as news, blogs, parks, and the like. The domain purpose may be the main purpose of the domain's website. This purpose may relate to the content of a particular website and may not necessarily pertain to the company that owns it. That is, the corporate website of a given company may have a different purpose than, for example, a promotional blog site for the company.

After the domain categories have been determined, the exemplary method includes storing the results on a computer-readable storage medium, displaying the results to a user, or electronically delivering the results to the requestor via an electronic communication network in step S1900. At least one may proceed. Embodiments perform targeted searches within stored data, and / or repeatedly perform the described methods on a range of domains, and provide results for historical analysis of the domain domain involved, as further described below. It may further include compiling.

4 shows an example of a high-level process flow for a monthly recurring cycle in accordance with embodiments of the present invention. This generally consists of inputs 610, components 620, and outputs 630. Additional tools 690 may be included. These additional tools may assist the user in performing various selections / inputs 610, for example, in prioritizing identified hyperlinks, terms, and the like. The inputs may include, for example, zone file 612, signature marker set 614, and training set 618. In embodiments, the user may enter or select items within signature marker set 614 and / or training set 618. The components can include a web crawler 622 and an analyzer 624. The analyzer can be functionally divided into categorization 626 and classification 628 portions. The registry 616 can maintain various processes (inputs, components, and outputs) and implement them on designated portions of the web space 650 to better understand the related online activities by the industry. For example, the registry can collect all DNS traffic from a given server. For the TLD, data for all sub-domain names may be collected periodically and maintained in the file store 632 for a period of time. Reports 634 may be generated based on a method including a training set 616 working in conjunction with additional tools 690 to process information from a designated portion of web space 650. This can provide DNS traffic values, domain status, objectives and categories for each domain name for a period of time, and provide access to information that was previously unavailable.

Also, embodiments may include receiving a set of domains input from a user for analysis as part of the query at step S2000. The system can automatically identify the attributes that are common between the input domains. These attributes may come from contextually matched content or other collected information. The results of this analysis may include outputting the common attributes identified in step S2100 to the user. This capability can provide advantages such as automatically identifying common purpose-related attributes of domains, including additional verified pages.

Also, embodiments may include receiving a set of purposes and / or categories, for example, entered from a user as part of a query in step S2000. Based on the described methods, domains corresponding to the entered set of purposes and / or categories can be identified, and the identified domains can be output to the user in step S2100. This capability may be advantageous in providing improved classification and identification of relevant information and / or domains that would not simply be possible from conventional methods of analyzing web page content. The described methods provide for prioritizing and dividing downloaded content, identifying and collecting significant attributes about the domain, including various categorization of direct and indirect content, and allowing users or administrators to Can be searched based on the attributes of.

Further details regarding the example process flow are provided in FIG. 6. As shown in FIG. 6, the process starts at S600 and proceeds to S610 where a target zone file is obtained. For example, the target zone can be a designated domain as described herein. The method proceeds to S620.

In S620, the process attempts to connect to the target zone, which may be the designated domain. Different types of errors can be caused based on attempts to connect to the target zone. For example, there may be lame delegation that the DNS server cannot run. If no DNS is resolved or other errors similar to those described above occur at S620, the method proceeds to S624, which is referred to as a "no DNS" error. Based on this determination, the status can be reported at S680.

If the attempt to connect to the target zone is successful, the method continues to S630. Successful attempts may include resolving the requested domain name to an IP address by a DNS server. However, there are errors that can occur at the web server level that can prohibit getting content from the requested address. For example, in S634, a server error may be identified, such as indicating a name server timeout or an error associated with the server in response to a request to connect to an IP address. If a server error is indicated at S634, the method may proceed to S680 where an error may be reported.

If the server is found by the server during S630, the method proceeds to S640-648 where various responses may be received based on an attempt to crawl the web site or address specified herein. This may include an indication that the requested domain does not have an active web site, as in S640. Also, as in S642, there may be an error in the web server responsible for the website displayed after the server is found. In addition, the server or website may limit the ability of the web crawler to retrieve content, as in S644, or redirect the web crawler to another site, as in S646. These, and other responses indicating less than maximum access to the content of the web site, may be reported in S680.

If the web site is accessed and the content is available, the method proceeds to S648 where it is recognized that the content has been found from the web site. As further described herein, once content is found from a web site, or from a target domain, the method may continue by accessing and analyzing the found content, as in S660. The results of the content search and / or analysis may be reported at S680.

Thus, the method shown in FIG. 6 may cause a number of various reports to appear in S680 depending on the degree to which the method proceeds to attempt to access the target zone. Some of these may reflect additional states, purposes, and categories depending on the amount and type of information, including content, obtained during the process, or non-functional state of the domain, such as in the case of DNS or web server errors.

Embodiments of the present invention may include systems implementing the described methods as well as computer-readable storage media encoded with instructions for causing a computer to execute the described methods. For example, as shown in FIG. 5, the electronic system 100, including the processor, memory, and electronic communication device, may be configured to check the status of the first page associated with the DNS server 140. System 100 may represent a user computer system, wireless communication devices such as 120 and 170, subnetworks such as 130 and 190, a server, or any other network capable device with the necessary functional capabilities. System 100 may operate as part of, or separate from, a DNS server associated with a registry.

The first page may be received by the system 100 from the server 150 via an electronic communications network 170 FMF, such as the Internet. System 100 can identify a plurality of additional pages from the domain based on hyperlinks from the first page. System 100 may then check the status of additional pages via DNS server 140. The system 100 may prioritize hyperlinks based on comparison with predetermined data, as described above. Content from at least one of the first page and the plurality of additional pages may be extracted via servers 150, 160, for example web-hosting servers. The system 100 may process the content via a set of signature markers stored on or otherwise accessed by the system 100 to determine a contextual match.

In addition, the system 100 may determine the purpose of the domain according to the state of the first page, the state of the additional page, and the content processing results. System 100 may receive various user inputs as described above, such as selecting a verified web page from which to extract data content, selecting terms, and the like, and as a result of the described processing performed by system 100. These can be displayed, stored and / or sent in accordance with known techniques.

System 100 includes storage that includes first storage (not shown, typically random access memory, or "RAM"), second storage (not shown, typically read-only memory, or "ROM"). Any number of processors (not shown) coupled to the devices. Both of these storage devices can include any suitable type of computer-readable media described and / or mentioned above. Mass storage devices (not shown) may be used to store programs, data, and the like, and are typically secondary storage media such as hard disks that are slower than primary storage. It will be appreciated that the information retained in the mass storage device may be included as virtual memory in a standard manner as part of main storage, where appropriate. Certain mass storage devices, such as CD-ROMs, may pass data in one direction to the processor.

System 100 includes other computers 110, such as video monitors, track balls, mice 104, keyboards, microphones, touch-sensitive displays, transducer card readers, magnetic or paper. It may include an interface that includes one or more input / output devices, such as a tape reader, a tablet, a stylus, a voice or handwriting recognizer, or other known input devices. System 100 may be coupled to a computer or other telecommunications network 170, 180 using a network connection as shown generally at 101. The network is a variety of wired, optical, electronic, and other known technologies for exchanging information between computers 110, servers 160, wireless communication devices 120, 170, and sub-networks 190, 130. You can connect networks. With this network connection, the system 100 and the processor therein may receive information from the network or may output information to the network in the course of performing the steps of the method described above. The devices and materials described above will be familiar to those skilled in computer hardware and software and need not be individually depicted one after another to those skilled in the art. The hardware elements described above may be configured to act as one or more modules to perform the operations described above (generally temporarily).

In addition, embodiments of the present invention further include a computer-readable storage medium including program instructions for performing various computer-implemented operations. In addition, the medium may include alone or in combination with program instructions, data files, data structures, tables, and the like. The media and program instructions may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind available to those skilled in the computer software arts. Examples of computer-readable storage media include magnetic media such as hard disks, floppy disks, and magnetic tape; Optical media such as CD-ROM disks; Magneto-optical media such as flow optical disks; And hardware devices, such as read-only memory devices (ROM) and random access memory (RAM), specially configured to store and execute program instructions. Examples of program instructions include, for example, machine code generated by a compiler, and files containing high level code that can be executed by a computer using an interpreter.

The invention has been described with reference to exemplary embodiments. Modifications and variations of the described embodiments may be apparent to those skilled in the art upon reading and understanding this specification. The present invention includes all such modifications and changes as long as they fall within the scope of the appended claims or their equivalents.

Claims (21)

(a) checking the status of the first web page associated with the domain name in an attempt to resolve the first web page;
(b) determining the purpose of the domain name based on the state of the first web page if the attempt to resolve the first web page causes an error;
(c) proceeding to steps (d) through (j) if the attempt to resolve the first web page is successful;
(d) retrieving a first web page of the domain name through a computer communication network;
(e) identifying a number of additional web pages related to the domain name based on hyperlinks retrieved from the first web page;
(f) identifying the status of the additional page in an attempt to resolve the additional web page;
(g) prioritizing the hyperlinks retrieved from the first web page through comparison with predetermined data;
(h) extracting content from the first page or from at least one of the plurality of additional pages, the priority of which is selected by priority of the additional pages over a computer communication network;
(i) processing the content through a predetermined table of related data elements to determine contextual matches present in the content;
(j) determining the purpose of the domain name by the status of the first web page, the status of the additional web page, and the contextual match identified through the content processing; And
(k) performing at least one of displaying, storing and transmitting the determined purpose of the domain;
Method for automating domain name profiles, including
Checking the status of the first page associated with the domain;
Retrieving the first page;
Identifying a number of additional pages from the domain based on the hyperlinks from the first page;
Checking the status of additional pages;
Prioritizing the hyperlinks by comparison with predetermined data;
Extracting content from the first page or from at least one of the plurality of additional pages, at least one of the additional pages being selected by priority;
Processing the content through a set of signature markers to determine a contextual match;
Determining the purpose of the domain name according to the state of the first page, the state of the additional page, and the content processing result; And
Performing one of displaying, storing and transmitting the determined purpose of the domain;
Automatic analysis method of domain name profiles including
3. The method of claim 2, wherein determining the purpose of the domain is further based on at least one of a domain name determined in the content, a registered sales marker, the absence of a particular data type, another publisher of the page, a social group identifier, and a data type. Method characterized by
3. The method of claim 2, further comprising: identifying one or more domains based on the purpose; And
Identifying individual categories of domains by their respective purposes and determining the categories of identified domains;
Method characterized in that it further comprises
5. The method of claim 4, further comprising identifying and searching related domains based on the determined category and preferred purpose of the domain.
3. The method of claim 2, further comprising verifying and searching for related domains based on the determined status and purpose of the domain.
3. The method of claim 2, wherein the content comprises executable code that includes a set of signature markers shaped to determine contextual conformance inherent in the first page or additional pages.
3. The method of claim 2, wherein the content comprises image, video and audio information comprising a set of signature markers shaped to determine contextual inherent in the first page or additional page.
3. The method of claim 2, wherein following the hyperlinks from the first page includes a traversing method that returns to the https version of the web page.
3. The method of claim 2, further comprising: receiving a set of domain inputs from a user;
Identifying attributes common to the input domains among contextually matched content; And
Outputting the identified attributes to the user;
Method characterized in that it further comprises
3. The method of claim 2, further comprising: receiving a set of purpose inputs from a user;
Identifying a domain corresponding to the set of purposes entered; And
Outputting the verified domain to the user;
Method characterized in that it further comprises
5. The method of claim 4, further comprising: receiving a set of purpose and category inputs from a user;
Identifying domains corresponding to a set of purposes and categories entered; And
Outputting the verified domain to the user;
Method characterized in that it further comprises
The method of claim 2,
Iii) attempting to resolve the domain name of the first page;
If the domain name resolution fails, confirming the state of the domain as inoperable;
If the domain name is resolved but the content is inaccessible, checking the status of the first page comprising the step of checking the status as inaccessible; And
Ii) attempting to resolve the domain name or network address of the first additional page;
If the resolution of the domain name or network address fails, verifying the status of the first additional page as non-functional;
If the domain name resolves but the content is inaccessible, confirming the status of the additional page comprising the step of verifying that the status of the first additional page is inaccessible;
Method characterized in that it further comprises

Checking the status of the first page associated with the domain;
Retrieving the first page;
Identifying a number of additional pages from the domain based on hyperlinks from the first page;
Checking the status of additional pages;
Prioritizing the hyperlinks by comparison with predetermined data;
Extracting content from the first page or from at least one of the plurality of additional pages, at least one of the additional pages being selected by priority;
Processing the content through a set of signature markers to determine a contextual match;
Determining the purpose of the domain name according to the state of the first page, the state of the additional page, and the content processing result; And
Performing one of displaying, storing and transmitting the determined purpose of the domain;
A computer-readable storage medium storing suitable instructions executable by a computer for performing a
15. The method of claim 14, wherein determining the purpose of the domain is further based on at least one of a domain name determined in the content, a registered sales marker, the absence of a particular data type, another publisher of the page, a social group identifier, and a data type. Computer-readable storage medium characterized by
15. The method of claim 14, wherein the command further comprises: identifying one or more domains based on the purpose; And
Identifying individual categories of domains by their respective purposes and determining the categories of identified domains;
And a computer-readable storage medium further comprising
15. The computer-readable storage medium of claim 14, wherein the content comprises executable code comprising a set of signature markers shaped to determine a contextual match inherent in a first page or additional page.
15. The computer-readable medium of claim 14, wherein the content includes image, video, and audio information comprising a set of signature markers shaped to determine contextual inherent in the first page or additional page. Storage media
The method of claim 16, wherein the instructions are:
Receiving a set of domain inputs from a user;
Identifying attributes common to the input domains among contextually matched content; And
Outputting the identified attributes to the user;
And a computer-readable storage medium further comprising
The method of claim 14,
Iii) attempting to resolve the domain name of the first page;
If the domain name resolution fails, confirming the state of the domain as inoperable;
If the domain name is resolved but the content is inaccessible, checking the status of the first page comprising the step of checking the status as inaccessible; And
Ii) attempting to resolve the domain name or network address of the first additional page;
If the resolution of the domain name or network address fails, verifying the status of the first additional page as non-functional;
If the domain name resolves but the content is inaccessible, confirming the status of the additional page comprising the step of verifying that the status of the first additional page is inaccessible;
And a computer-readable storage medium further comprising
An electronic communication unit configured to exchange data via an electronic network;
Means for checking the status of a first page related to the domain;
Means for retrieving the first page;
Means for identifying a number of additional pages from a domain based on hyperlinks from the first page;
Means for checking the status of the additional page;
Means for prioritizing the hyperlinks based on a comparison with predetermined data;
Means for extracting content from the first page or from at least one of the plurality of additional pages in which at least one of the additional pages is selected by priority;
Means for processing the content via a signature marker set to determine contextual matches;
Means for determining the purpose of the domain according to the state of the first page, the state of additional pages, and the content processing result; And
At least one of a display and a storage device for displaying or storing the determined purpose of the domain.

Images