US20160269248A1 - Systems and methods for detecting, identifying and categorizing intermediate nodes - Google Patents
Systems and methods for detecting, identifying and categorizing intermediate nodes Download PDFInfo
- Publication number
- US20160269248A1 US20160269248A1 US15/162,255 US201615162255A US2016269248A1 US 20160269248 A1 US20160269248 A1 US 20160269248A1 US 201615162255 A US201615162255 A US 201615162255A US 2016269248 A1 US2016269248 A1 US 2016269248A1
- Authority
- US
- United States
- Prior art keywords
- node
- data
- source
- source node
- candidate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/14—Network analysis or design
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/562—Brokering proxy services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/954—Navigation, e.g. using categorised browsing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/12—Discovery or management of network topologies
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1441—Countermeasures against malicious traffic
- H04L63/1466—Active attacks involving interception, injection, modification, spoofing of data unit addresses, e.g. hijacking, packet injection or TCP sequence number attacks
Definitions
- Various countries, corporations and Internet Service Providers block, censor or filter communications transmitted between two or more nodes. These communications can occur via Internet, Extranet, Intranet or any other communication path that allows two nodes to communicate with one another.
- the type of communication is further independent of communication path and includes, but is not limited to, client-server, peer-to-peer and mainframe architectures. All types of communications, including but not limited to wireless, cellular, wired, optical and satellite communications may be subject to censorship. Moreover various modes of communication including, but not limited to, client-server, mainframe, distributed and peer-to-peer, are subject to censorship.
- a given Intermediate Node 200 can cache obtained Target Content 300 and still be considered an Intermediate Node 200 as long as the Requesting Node 100 is attempting to obtain data from the Responding Node 1400 .
- the data may be as simple as a low level communications request to check if a target server exists, or the data may be as complex as is supported on the communication path used and by the type of communications selected.
- an end user is not required. Automated machine-to-machine communications, routing between systems, networking devices and other communication-related efforts may utilize an Intermediate Node 200 in place of an end user.
- An end user can, therefore, be a human, a computer, a program or some portion of code that produces a Node Request 400 .
- Node Requests 400 may be generated directly or indirectly with or without knowledge of the Intermediate Node 200 .
- Content Requests 500 need not be defined as distinct or separate from the Node Requests 400 , because the Content Request 500 can be a routed Node Request 400 or a context-based new message.
- This process including the Node Request 400 , the Content Request 500 , the Content Response 600 and the Intermediate Node Response 700 , may be cached by the Intermediate Node 200 , and the Intermediate Node 200 may modify, possibly in a malicious manner, the contents of the Target Content 300 prior to sending the content back to the Requesting Node 100 .
- One challenge associated with the conventional use of Intermediate Nodes 200 is that the Requesting Node 100 typically does not have any visibility into the Content Request 500 and the Content Response 600 performed between the Intermediate Node 200 and the Responding Node 1400 . This lack of visibility enables certain nefarious Intermediate Nodes 200 to promote services to Requesting Nodes 100 that the Intermediate Nodes 200 mayor may not provide. Even if a Requesting Node 100 can determine where an Intermediate Node 200 is located based on information such as a Uniform Resource Indicator (URI) of the Intermediate Node's 200 access point, there is no guarantee that this is the location that a Responding Node 1400 containing the Target Content 300 sees from the Intermediate Node 200 . Many Intermediate Nodes 200 use one physical node to accept incoming Node Requests 400 and a completely different node to send out Content Requests 500 .
- URI Uniform Resource Indicator
- the Node Crawler 800 obtains the Candidate Node Data 1700 for a particular Source Node 1600 , the data for the particular Source Node 1600 (e.g., an Intermediate Node 200 ) corresponding thereto is sent to the Logical Data Repository 1200 .
- the Candidate Node Data 1700 provides sufficient information to interact with a given node, but it is not restricted to only Intermediate Nodes 200 .
- the Node Crawler System 1100 may extract Candidate Node Data 1700 for each Source Node Data 1650 /Source Node 1600 and send the extracted data for each Source Node 1600 to the Logical Data Repository 1200 .
- the Node Crawler System 1100 may directly utilize the Candidate Node Data 1700 without subsequent processing through a separate node from the Logical Data Repository 1200 .
- the Node Crawler System 1100 may be run on a given network to identify all physical devices connected to that network.
- the Source Node Data 1650 in this case may include computer specifications, for example, operating system information, hardware information, accessible ports and location within the network.
- the Node Crawler System 1100 may be utilized to dynamically determine all nodes within the network and properly categorize each node in turn.
- each identified logical node may be further inspected by the Node Crawler System 1100 until all physical instances for each logical node have been defined and extracted as Candidate Node Data 1700 .
- Data may be extracted by the Web Crawler 800 from any type of node including, for example, File Systems 2600 , local, remote, federated or distributed Logical Data Repositories 2700 or from any Responding Node 1400 .
- Data may be entered into the Node Crawler System 1100 or directly into the Logical Data Repository 1200 through any of these options or through any integration layer, e.g., data integration, business layer options and presentation layer features (e.g., scraping, mash-up technology or similar options).
- the Intermediate Node Verification System 1500 uses a Requesting Node 100 in a controlled manner to initiate this testing and determination process.
- Control of the Requesting Node 100 i.e., Controlled Requesting Node 1300
- Controlled Requesting Node 1300 is obtained by, for example, hardware which interacts with all communications from a given device; an external communications option, e.g., a firewall; or utilizing software, code components or services on a node that interact with all communications for a given test.
- the Controlled Requesting Node 1300 does not need a component that interacts with all communications, although the Controlled Requesting Node 1300 can interact with all communications in some embodiments. Rather, the Controlled Requesting Node 1300 need only be able to target communications for a given test of an Intermediate Node 200 .
- the target communications are defined in more detail below.
- the Controlled Requesting Node 1300 is configured such that the Intermediate Node Verification System 1500 can use the Candidate Node Data 1700 to generate a Node Request 400 from the Controlled Requesting Node 1300 .
- the Intermediate Node Verification System 1500 may further obtain and store the Intermediate Node Response 700 .
- Any combination of the Node Request 400 , the Intermediate Node Response 700 and optional tests run on the logical Controlled Requesting Node 1300 are collectively referred to as Requesting Node Data 1800 .
- the Controlled Requesting Node 1300 sends a Node Request 400 for Target Content 300 to a Controlled Responding Node 1450 based on the Candidate Node Data 1700 for the Intermediate Node 200 to be tested.
- the Controlled Responding Node 1450 may be any logical node that contains the Target Content 300 .
- Control of the Responding Node 1400 i.e., Controlled Responding Node 1450 , is obtained by, for example, hardware which interacts with all communications from a given device; an external communications option, e.g., a firewall; or utilizing software, code components or services on a node that interact with all communications for a given test.
- the Node Verification System 1500 utilizes a known set of data sent from the Controlled Responding Node 1450 to the Controlled Requesting Node 1300 to discern extra content inserted by the Intermediate Node 200 .
- the Node Verification System 1500 can utilize manual text inspection, Regex Expressions, workflow processes or other similar techniques to discern how the data from the Controlled Responding Node 1450 is being included in the overall Content Response 600 from the Intermediate Node 200 .
- the Controlled Responding Node 1450 creates contextual Content Responses 600 , which include content related to the incoming request, and stores nothing.
- the Controlled Requesting Node 1300 and Controlled Responding Node 1450 directly communicate some combination of Requesting Node Data 1800 and Responding Node Data 1900 through any available communication type or path.
- the Controlled Requesting Node 1300 sends out Node Requests 400 but stores no information.
- the content of the requests and responses from the Controlled Requesting Node 1300 , the Intermediate Node 200 and the Controlled Responding Node 1450 are ignored and, instead transmission times, amongst other options are utilized to determine functionality and characteristics of the Intermediate Node 200 .
Abstract
A system and method for obtaining node information from a variety of potential sources and storing the information in a logical repository, and a system and method for identifying and categorizing Intermediate Nodes using a combination of requesting and responding node information.
Description
- The present application claims priority to U.S. application Ser. No. 13/465,799 filed May 7, 2012, entitled “Systems and Methods for Detecting, Identifying and Categorizing Intermediate Nodes,” which is incorporated herein by reference in its entirety for all purposes.
- The present disclosure relates to obtaining information for intermediate nodes through which target content can be obtained and, in particular, to systems and methods for detecting, identifying and categorizing intermediate nodes, including determining the type and capabilities of intermediate nodes.
- Web tracking solutions can generally be separated into solutions loaded into a customer's server, for example, packet “sniffing” and IIS log file analysis software, and solutions that attempt to track page level activity and which take the form of code inserted on a page, third party Web “cookies” or software applications.
- Various countries, corporations and Internet Service Providers block, censor or filter communications transmitted between two or more nodes. These communications can occur via Internet, Extranet, Intranet or any other communication path that allows two nodes to communicate with one another. The type of communication is further independent of communication path and includes, but is not limited to, client-server, peer-to-peer and mainframe architectures. All types of communications, including but not limited to wireless, cellular, wired, optical and satellite communications may be subject to censorship. Moreover various modes of communication including, but not limited to, client-server, mainframe, distributed and peer-to-peer, are subject to censorship.
- For example, a user may subscribe to an Internet sports package to watch sporting events over a network. The user can request and watch so-called out of market games, but the games are often censored (referred to as “blacked out”) when the team is playing locally and the televised version of the game is available on local free or pay television channels. The distributor of the content identifies the source of the content request and denies the request when the source is within the blackout areas.
-
FIG. 1 shows a system including anIntermediate Node 200 utilized to send and receive requests for TargetContent 300.Intermediate Nodes 200, for example proxy servers, were created in part to overcome censorship.Intermediate Nodes 200 come in a variety of configurations, capabilities, uses and placement with a requirement that, at some point, they respond to a request for TargetContent 300 from a RequestingNode 100. Caching aside, theNode Request 400 forTarget Content 300 is not targeted at information typically stored locally by theIntermediate Node 200. Rather, the NodeRequest 400 is focused on information typically stored on yet another logical node, referred to herein as aResponding Node 1400, which is physically or logically separate from theIntermediate Node 200. The TargetContent 300 can be any content, for example, a service, a file, a connection, a web page, multimedia or any other resource available over a network. - As another example, a user living in Los Angeles, Calif., representing a possible Requesting Node 100 may normally be blocked from obtaining
Target Content 300, e.g., online TV, from a specific website which represents a RespondingNode 1400, because that representative website is configured to only serve content to users in the state of New York. Referring toFIG. 1 , the user (at Requesting Node 100) may find anIntermediate Node 200 which requests the content from the RespondingNode 1400 from within the state of New York. The user sends a request for the content on the target website to the IntermediateNode 200, and theIntermediate Node 200 obtains the content, unrestricted in this example, from the target website and returns the content to the user in Los Angeles. - A given
Intermediate Node 200 can cache obtained TargetContent 300 and still be considered anIntermediate Node 200 as long as the RequestingNode 100 is attempting to obtain data from theResponding Node 1400. The data may be as simple as a low level communications request to check if a target server exists, or the data may be as complex as is supported on the communication path used and by the type of communications selected. - Nodes are logical constructs that can be physically implemented as a discrete node, as part of other logical nodes or as a system. Requesting
Nodes 100,Intermediate Nodes 200 and RespondingNodes 1400 may exist at the same physical location, at completely disparate physical locations or at any combination thereof. Logical nodes may be comprised of different parts of a larger system, be themselves independent systems or be combined together in any combination. For example, a group of networked computers may each utilize a shared access point that is, itself, acting on behalf of a single logical node. - Many
Intermediate Nodes 200 do not provide visibility to their data retrieval activities, and this lack of visibility causes difficulties with respect to the conventional use ofIntermediate Nodes 200. ManyIntermediate Nodes 200 do not provide the services that they purport to offer and, in fact, many nefariousIntermediate Nodes 200 cause more harm than any benefit they may provide. HarmfulIntermediate Nodes 200 may download malicious content onto a RequestingNode 100, infiltrate the RequestingNode 100 by utilizing an array of techniques or promote the location of the RequestingNode 100 to dangerous third party groups. The Requesting Node 100 has almost no inherent protection from harmfulIntermediate Nodes 200. - Moreover, using an
Intermediate Node 200 through any sort of manual effort can be both technically challenging and time consuming for a typical end user.Intermediate Node 200 usage may require entries to be made in special sections of a Requesting Node's 100 operating system, file directory or some other configuration option, either directly or indirectly, and the only manner in which to determine if anIntermediate Node 200 is a viable and functional option is typically to use theIntermediate Node 200 and hope that nothing harmful occurs to the RequestingNode 100. Given the large number ofIntermediate Nodes 200 providing intermittent connectivity, an end user may have to attempt to use hundreds or more ofIntermediate Nodes 200 prior to finding a somewhat viable option. - Compounding these problems with the conventional use of
Intermediate Nodes 200 is that an apparently functionalIntermediate Node 200 may hide additional data within theTarget Content 300 or perform actions beyond the scope of theResponding Node 1400 that can directly or indirectly affect the RequestingNode 100. While an end user may find an apparently functionalIntermediate Node 200, through which requests for TargetContent 300 are fulfilled, the end user may have no idea if theIntermediate Node 200 is also downloading malicious content or performing other potentially harmful operations. Furthermore, the end user has no way of knowing from which geographic region a givenIntermediate Node 200 is sending outContent Requests 500 to theResponding Node 1400. Overcoming censorship may rely on being perceived as requesting information from a distinct and safe geographic region but, given the conventional options in the market, choosing a specific location for anIntermediate Node 200 is not possible. - It should be noted that an end user is not required. Automated machine-to-machine communications, routing between systems, networking devices and other communication-related efforts may utilize an
Intermediate Node 200 in place of an end user. An end user can, therefore, be a human, a computer, a program or some portion of code that produces aNode Request 400.Node Requests 400 may be generated directly or indirectly with or without knowledge of theIntermediate Node 200.Content Requests 500 need not be defined as distinct or separate from theNode Requests 400, because theContent Request 500 can be a routedNode Request 400 or a context-based new message. - The present disclosure provides a system and method that protects Requesting Nodes from harmful Intermediate Nodes while allowing Requesting Nodes to determine the functionality and location of Intermediate Nodes.
- In accordance with one embodiment of the present disclosure, a Node Crawler System utilizes a variety of data conduit options to obtain Candidate Node Data for different types of Intermediate Nodes from Source Nodes. The Candidate Node Data may be stored in a Logical Data Repository.
- In another embodiment, an Intermediate Node Verification System utilizes Candidate Node Data to generate Node Requests that enable data collected at the Requesting Node and the Responding Node. The collected Requesting Node Data and Responding Node Data are used to analyze and overcome attacks used by harmful Intermediate Nodes. The analysis of the Node data results in information about each Intermediate Node that may be used to determine various features and the relative safety of using a particular Intermediate Node.
- Other objects and features of the present disclosure will become apparent from the following detailed description considered in conjunction with the accompanying drawings. It is to be understood, however, that the drawings are intended solely for purposes of illustration and not as a definition of the limits of the disclosure, for which reference should be made to the appended claims.
- In the drawings, wherein like reference numerals delineate similar elements:
-
FIG. 1 is a schematic diagram that shows a system including anIntermediate Nodes 200 utilized to send and receive requests forTarget Content 300; -
FIG. 2 is a schematic drawing that illustrates a NodeCrawler System 1100 and aNode Verification System 1500 according to example embodiments; -
FIG. 3 shows types of Candidate Node Data 1700 according to example embodiments; -
FIG. 4 is a schematic drawing showing input options for Candidate Node Data 1700 into aLogical Data Repository 1200 according to example embodiments; -
FIG. 5 is a schematic drawing of an optionalRule Generation Tool 2800 according to an example embodiment; and -
FIG. 6 is a schematic drawing demonstrating further logical node constructs of the Node Crawler System 1100 and the IntermediateNode Verification System 1500 according to example embodiments. - The present disclosure provides systems and methods for identifying, geo-locating and categorizing
Intermediate Nodes 200. This discussion provides a general overview of example embodiments prior to describing each in more detail below. -
FIG. 2 is a schematic drawing that illustrates aNode Crawler System 1100 and aNode Verification System 1500 according to example embodiments. - The
Node Crawler System 1100 obtains and submits Candidate Node Data 1700 to aLogical Data Repository 1200. As explained in more detail in subsequent sections below, theNode Crawler System 1100 obtains data through various mechanisms over any type of communication path, using any type of communication, and can either pull Candidate Node Data 1700 fromSource Nodes 1600 or receive Candidate Node Data 1700 fromSource Nodes 1600. - The Intermediate
Node Verification System 1500 utilizes Candidate Node Data 1700 to determine the functionality ofIntermediate Nodes 200. The IntermediateNode Verification System 1500 comprises a Controlled RequestingNode 1300 and a Controlled RespondingNode 1450 to generate a communication path through a targetIntermediate Node 200. Data on the targetIntermediate Node 200 is collected at the Controlled RequestingNode 1300 and the Controlled RespondingNode 1450. The collected data for theIntermediate Node 200 is used by the IntermediateNode Verification System 1500 to categorize a type of theIntermediate Node 200 and to determine the safety and functional capabilities of theIntermediate Node 200. - Communications between nodes and between nodes and other system elements can occur via any viable means including, but not limited to, wired, wireless, cellular, optical and satellite communications. The communications can occur via the Internet, an Extranet, Intranet or any other network type that provides access through two or more nodes. A mode of the communications is not limited and may include, for example, peer-to-peer, client-server and mainframe architectures.
- A. Node Verification System
-
FIG. 1 shows a system including anIntermediate Node 200 utilized by a RequestingNode 100 to obtainTarget Content 300 from a RespondingNode 1400.Target Content 300 includes any type or form of data. TheTarget Content 300 may include text data or binary data or some combination thereof, and may be obtained through any viable means including, for example, Internet Requests, Web Service calls, FTP, SMTP, UDP and TCP. - The Requesting
Node 100 sends aNode Request 400 to theIntermediate Node 200 either directly or indirectly through a method such as an HTTP 302 Redirect or a TCP Resend message. After theIntermediate Node 200 receives theNode Request 400, theIntermediate Node 200 may request theTarget Content 300 from the RespondingNode 1400 through aContent Request 500. - To the Responding
Node 1400, theContent Request 500 may look as if it has been sent from theIntermediate Node 200, and the RespondingNode 1400 sends theTarget Content 300 back to theIntermediate Node 200 via astandard Content Response 600. It should be noted that theIntermediate Node 200, depending on a type of theIntermediate Node 200, may provide identifying information about the RequestingNode 100 to the RespondingNode 1400. TheIntermediate Node 200 returns theTarget Content 300 to the RequestingNode 100 via anIntermediate Node Response 700. This process, including theNode Request 400, theContent Request 500, theContent Response 600 and theIntermediate Node Response 700, may be cached by theIntermediate Node 200, and theIntermediate Node 200 may modify, possibly in a malicious manner, the contents of theTarget Content 300 prior to sending the content back to the RequestingNode 100. - As with other nodes, an
Intermediate Node 200 is a logical construct that may be physically implemented on the same physical node as the RequestingNode 100 and/or the RespondingNode 1400, or on a separate physical node. TheIntermediate Node 200 may be physically implemented as a series ofIntermediate Nodes 200, an open or closed system of optionally distributed nodes or some variation thereof. SomeIntermediate Nodes 200 may provide access in one geographic location and send outContent Requests 500 through one or more geographic locations, all or some of which are different from the geographic location of the access point. - One challenge associated with the conventional use of
Intermediate Nodes 200 is that the RequestingNode 100 typically does not have any visibility into theContent Request 500 and theContent Response 600 performed between theIntermediate Node 200 and the RespondingNode 1400. This lack of visibility enables certainnefarious Intermediate Nodes 200 to promote services to RequestingNodes 100 that theIntermediate Nodes 200 mayor may not provide. Even if a RequestingNode 100 can determine where anIntermediate Node 200 is located based on information such as a Uniform Resource Indicator (URI) of the Intermediate Node's 200 access point, there is no guarantee that this is the location that a RespondingNode 1400 containing theTarget Content 300 sees from theIntermediate Node 200.Many Intermediate Nodes 200 use one physical node to acceptincoming Node Requests 400 and a completely different node to send out Content Requests 500. - “URI” is an acronym for Universal Resource Identifier and is a term of art used to denote any set of data sufficient to locate a node on a given type of communications. Nodes can support different URI values for different types of communication and URI values can exist across different communication paths and modes. Examples of URI values include IP Addresses, IP Addresses and Ports, URL values, TCP addresses, email address and Domain Name Services (DNS) entries in a DNS Name Server. A Port is an optional value that enables a given node to support different requests, or types of communications, using the same IP address. It should be noted that the above examples of URI values are only a few possible examples of URI values, and the concept of a universal identifier is not limited to these examples.
- To overcome the lack of visibility between the
Intermediate Node 200 and the RespondingNode 1400, as well as other challenges, example embodiments of the present disclosure provide aNode Crawler System 1100, as shown inFIG. 2 , which obtains Candidate Node Data 1700 fromSource Nodes 1600. While the schematic drawing inFIG. 2 shows theNode Crawler System 1100 and theNode Verification System 1500 as existing on different nodes, this configuration is presented for sake of clarity. In other example embodiments, theNode Crawler System 1100 and theNode Verification System 1500 exist on the same node or, at the least, on tightly integrated nodes that share some form of memory, e.g.,Logical Data Repository 1200. - The
Logical Data Repository 1200 is a logical construct that can be physically implemented as shared memory and may be, for example, a memory file, a shared queue, a web service call or a file system storage. TheLogical Data Repository 1200 may be a single data management solution or a database, or theLogical Data Repository 1200 may be part of a larger set of repositories either on the same physical node or on separate physical nodes located in various geographical locations. TheNode Crawler System 1100, theLogical Data Repository 1200 and theNode Verification System 1500 may be implemented as a single physical node or as three systems of a multitude of nodes or some variation thereof. The arrows showing data flow between various nodes and theLogical Data Repository 1200 inFIG. 2 denote direction of data and do not show or place any restrictions on a number of types of connections thereto. In some example embodiments, a given node sends or receives data from one node within aLogical Data Repository 1200 or from multiple nodes. - Communications with the Logical Data Repository may occur serially or in parallel and may occur across different types of communication and different communication paths, or the communications may occur on a single type of communication or a single type of communication path or any combination thereof. There is no restriction that data flowing in one direction must flow in the other direction using the same communication types or paths. Thus, data may flow from the
Logical Data Repository 1200 to a given node using one path, or sets of paths, and utilizing a given type, or types, of communication, and that second node can send data back to theLogical Data Repository 1200 using the same or completely different types and paths of communication. Furthermore, theLogical Data Repository 1200 and a given node can use a single mode of communication or multiple modes of communication, and the modes used can extend over a plurality of communication types and paths. - The
Node Crawler System 1100 comprises a set ofSource Node 1600 information including at least one target Uniform Resource Identifier (URI). TheSource Node 1600 information itself may be stored anywhere, including theLogical Data Repository 1200 and one or more of a plurality of local data stores, text files, XML files, web service sources or any other source in any combination thereof that can return the at least one of a possible plurality of URI values. A given URI points to aSource Node 1600 that is on the same node as theNode Crawler 800 or external to the logical node of theNode Crawler 800. TheSource Node 1600 providesSource Node Data 1650 that comprises, for example, text data or binary data or some combination thereof, and can be stored in a structured or unstructured manner. - The
Node Crawler 800 obtains theSource Node Data 1650 for a givenSource Node 1600 based on a given URI. TheNode Crawler 800 may obtainSource Node Data 1650 for each of the Source Nodes 1600 (i.e., the URIs) in the set of source node information or for a portion thereof. TheNode Crawler 800 may obtainSource Node Data 1650 for each of theSource Nodes 1600 of a given network or for a portion thereof. TheNode Crawler 800 may obtainSource Node Data 1650 forSource Nodes 1600 selected based on web services, applications and/or search results. TheNode Crawler 800 may optionally apply rules found in the Dynamic Rule Repository 1000 to extract possible Candidate Node Data 1700 from theSource Node Data 1650. For example, theNode Crawler 800 may iterate through the rules found in the Dynamic Rule Repository to extract the Candidate Node Data 1700 for the givenSource Node 1600. Based on the types ofIntermediate Nodes 200 being targeted, the actual Candidate Node Data 1700 formats can look considerably disparate in nature. - For example, as shown in
FIG. 3 ,certain Intermediate Nodes 200 require an IP Address (e.g., 1.1.1.1) along with a port (e.g., 3128) to work properly.Other Intermediate Nodes 200 require a main domain name (e.g., a URL, for example, http://www.myNode.com) to be accessed. Still other types ofIntermediate Nodes 200 have specific naming conventions that apply only to the network in which they operate. Some Candidate Node Data 1700 is comprised of Communication Headers 2300, e.g., TCP header data or HTTP header information. The Candidate Node Data 1700 may be in the form of aDisconnected Data Set 2200,URI Data 2000 that is optionally extracted from text, HTML or other such content, orData Sources 2100, e.g., federated queried data, replicated data, encapsulated or transportable databases, text repositories or files, XML data or other sets of relational or non-relational data or other such data forms that contain sufficient data to access and utilize a givenIntermediate Node 200. - Some
Intermediate Nodes 200 require authentication information or various types and/or layers of encryption whereas others require a workflow process for ongoing interactions. Still other intermediate nodes provide different services based on varying configurations. All of these optional information features combine to create multiple optional formats of Candidate Node Data 1700. - Given this range of possible Candidate Node Data 1700, the
Node Crawler 800 may employ a dynamic set of rules which are iterated through in an effort to obtain possible candidate matches. The Dynamic Rule Repository 1000 may contain rules containing any commands that are viable for extracting data fromSource Node Data 1650 to create the Candidate Node Data 1700. In one embodiment, two types of commands are used for extracting data: Regular Expressions and Custom Commands. Regular Expressions refer to a term of art describing a well-established syntax and language for providing series of commands that are used to pattern match a phrase or series of characters in any set of data. There are well-known Regular Expression libraries and processors that provide an extremely flexible array of matching options. In one embodiment, as shown inFIG. 5 , an administrative tool provides an ability to manually test and enter Regular Expression patterns into the Dynamic Rule Repository 1000 for subsequent use. - When Regular Expressions are not sufficient, or when multiple Regular Expressions are required to be applied in a certain manner, the
Node Crawler 800 may employ Custom Commands. Custom Commands perform operations including, for example, Binary/Text Search and Replace; Binary/Text Bidirectional Conversions; Bitwise Comparison Operations; Expression/Command Workflow, Web Browser Emulation, Scripting Engine Methods and Language Translation processes. In another embodiment, the Custom Commands are stored as workflow processes which are themselves iterated through for eachSource Node Data 1650 being processed. The individual Regular Expressions, Custom Commands and Workflows are stored in the Dynamic Rule Repository 1000 as rules. - The Dynamic Rule Repository 1000 itself is a logical node construct and, as such, may exist within the same memory space as the
Node Crawler 800 or in a separate physical location, or some combination of the two, and may be comprised of multiple physical implementation options ranging from a portion of memory utilized by theNode Crawler 800 to a completely distributed system spread across a range of geographic regions. The Dynamic Rule Repository 1000 may store rules in a permanent manner, a transient manner or some combination thereof. - As a further example, in one embodiment the Dynamic Rule Repository may be encapsulated in a series of Dynamic Link Libraries (DLL) files that are utilized by the
Node Crawler 800 through a series of binary requests. In this embodiment, the logic, steps and options are built into the DLL files and treated as a discrete functional block by theNode Crawler 800. In this embodiment, proven techniques and rules might be used without a dynamic learning component for speed optimization purposes. - Rules may be generated through an optional
Rule Generation Tool 2800 as shown inFIG. 5 , wherein end users can manually create rules and store the created rules in the Dynamic Rule Repository 1000. The end users may enter a URI and obtainSource Node Data 1650 which may appear as, but is not limited to, text or may be binary data such as audio/video or a TCP data stream. The end user may apply various expressions and commands against theSource Node Data 1650 until the desired Candidate Node Data 1700 has been extracted therefrom. The custom commands include branching, saving and other such options to allow a single set ofSource Node Data 1650 to be parsed in various manners to extract different Candidate Node Data 1700 formats as needed. - The rules may be combined with rules obtained in a dynamic manner and transiently stored. In an example embodiment, a given set of
Source Node Data 1650 is inspected for URI values using predefined rules while the underlying TCP communications is parsed for TCP Header information leading to transient rules for accessingparticular Source Nodes 1600. - The
Node Crawler 800, after it has obtained theSource Node Data 1650 from aSource Node 1600 via a given URI, may successively iterate through the available rules. In one embodiment, all available rules are applied to every content instance to obtain as many matches as possible. In another embodiment, the remainder of the rules, after a first rule which obtains Candidate Node Data 1700 is utilized, are not applied, and that successful rule may be stored for subsequent content retrievals from that URI or URIs related to that URI. If a previously successful match fails to produce candidates in the subsequent content retrievals, the other patterns may be iterated through. - In still another further embodiment, rules are iterated through in a successive series of steps and automatically chained together to form increasingly complex processing logic. The rules may be basic rules focused on obtaining general components of information or removing nonessential pieces of data. For example, a rule for finding IP Addresses and Ports might remove all letters and HTML punctuation from the
Source Node Data 1650. Other possible rules may include rules for transforming strings into string arrays based on a variable set of delimiters, and yet other rules may only extract specific IP addresses or numbers. TheNode Crawler 800 may iterate through the rules to progressively filter out content until the desired IP Address and Ports are obtained. Successive iterations generate subsequent steps in a dynamically determined workflow for theSource Node Data 1650. - Client-side scripting libraries, code components, methods, DLLs or embedded code, amongst other options, may be used to parse incoming content including the
Source Node Data 1650. A goal of example embodiments may be to process incoming content such that the end result is similar to or exactly the same as what is presented through a standard web browser, e.g., Internet Explorer, Firefox or Chrome. Examples of client side scripting include, for example, JavaScript, VB Script, Action Script and AJAX. An example embodiment may include multiple request support to further load such features as images, iFrame/framed-in or layered content or any other synchronous/asynchronous or additional content that would be retrieved by a web browser for a given request. - The
Node Crawler 800 according to another embodiment utilizes image recognition software, matching technology or a manual matching process to transform images into text equivalents. For example, a givenData Source Node 1600 may provide a series of images representing port numbers asSource Node Data 1650. In this case, theRule Generation Tool 2800 shown inFIG. 5 may be utilized to enable a user to provide the text contained by the image as a new rule. TheNode Crawler 800 uses the rule to transform matching images into usable port numbers for Candidate Node Data 1700. - Each time the
Node Crawler 800 obtains the Candidate Node Data 1700 for aparticular Source Node 1600, the data for the particular Source Node 1600 (e.g., an Intermediate Node 200) corresponding thereto is sent to theLogical Data Repository 1200. The Candidate Node Data 1700 provides sufficient information to interact with a given node, but it is not restricted to onlyIntermediate Nodes 200. TheNode Crawler System 1100 may extract Candidate Node Data 1700 for eachSource Node Data 1650/Source Node 1600 and send the extracted data for eachSource Node 1600 to theLogical Data Repository 1200. - The
Node Crawler System 1100 may directly utilize the Candidate Node Data 1700 without subsequent processing through a separate node from theLogical Data Repository 1200. In a further embodiment, theNode Crawler System 1100 may be run on a given network to identify all physical devices connected to that network. TheSource Node Data 1650 in this case may include computer specifications, for example, operating system information, hardware information, accessible ports and location within the network. TheNode Crawler System 1100 according to this embodiment may be utilized to dynamically determine all nodes within the network and properly categorize each node in turn. As a still further embodiment, each identified logical node may be further inspected by theNode Crawler System 1100 until all physical instances for each logical node have been defined and extracted as Candidate Node Data 1700. - The
Node Crawler System 1100 may obtain Candidate Node Data 1700 in multiple different formats, as shown inFIG. 4 .Source Nodes 1600 may be of any type of node and can transmitSource Node Data 1650 to theNode Crawler System 1100 or be polled for said data. In some example embodiments,Web Services 2400 or Applications 2500 are used to sendSource Node Data 1650, or even Candidate Node Data 1700, directly to theNode Crawler System 1100. This data may already be parsed into the requisite Candidate Node Data 1700, or need to be parsed and/or transformed accordingly. The data may be complete or incomplete with rules being utilized to fill in missing components of data as required. In further embodiments, search engines may be utilized directly or indirectly through screen-scraping, API, web services or other such access to search and return results to theNode Crawler System 1100. Search terms used in the search engine processes may be entered by users, obtained from other crawling efforts or derived from prior search efforts. - Data may be extracted by the
Web Crawler 800 from any type of node including, for example,File Systems 2600, local, remote, federated or distributedLogical Data Repositories 2700 or from any RespondingNode 1400. Data may be entered into theNode Crawler System 1100 or directly into theLogical Data Repository 1200 through any of these options or through any integration layer, e.g., data integration, business layer options and presentation layer features (e.g., scraping, mash-up technology or similar options). - B. Intermediate Node Verification System
- Referring again to
FIG. 2 , the IntermediateNode Verification System 1500 utilizes Candidate Node Data 1700 to test whether a given node is anIntermediate Node 200 and, if so, to determine a type of theIntermediate Node 200 and the capabilities of theIntermediate Node 200 as well as the potential impact that using the node might have on a RequestingNode 100. - The Intermediate
Node Verification System 1500 uses a RequestingNode 100 in a controlled manner to initiate this testing and determination process. Control of the RequestingNode 100, i.e., Controlled RequestingNode 1300, is obtained by, for example, hardware which interacts with all communications from a given device; an external communications option, e.g., a firewall; or utilizing software, code components or services on a node that interact with all communications for a given test. It should be noted that the Controlled RequestingNode 1300 does not need a component that interacts with all communications, although the Controlled RequestingNode 1300 can interact with all communications in some embodiments. Rather, the Controlled RequestingNode 1300 need only be able to target communications for a given test of anIntermediate Node 200. The target communications are defined in more detail below. - The Controlled Requesting
Node 1300 is configured such that the IntermediateNode Verification System 1500 can use the Candidate Node Data 1700 to generate aNode Request 400 from the Controlled RequestingNode 1300. The IntermediateNode Verification System 1500 may further obtain and store theIntermediate Node Response 700. Any combination of theNode Request 400, theIntermediate Node Response 700 and optional tests run on the logical Controlled RequestingNode 1300 are collectively referred to as Requesting Node Data 1800. - The Controlled Requesting
Node 1300 may be any RequestingNode 100. In an example embodiment, the Controlled RequestingNode 1300 is a process in the same physical server as theNode Crawler 800. In other embodiments, theNode Crawler 800 is a system comprised on multiple nodes, and the Controlled RequestingNode 1300 is, itself, in a completely separate system. - The Controlled Requesting
Node 1300 sends aNode Request 400 forTarget Content 300 to a Controlled RespondingNode 1450 based on the Candidate Node Data 1700 for theIntermediate Node 200 to be tested. The Controlled RespondingNode 1450 may be any logical node that contains theTarget Content 300. Control of the RespondingNode 1400, i.e., Controlled RespondingNode 1450, is obtained by, for example, hardware which interacts with all communications from a given device; an external communications option, e.g., a firewall; or utilizing software, code components or services on a node that interact with all communications for a given test. It should be noted that the Controlled RespondingNode 1450 does not need a component that interacts with all communications, although the Controlled RespondingNode 1450 can interact with all communications in some example embodiments. Rather the Controlled RespondingNode 1450 need only be able to interact with target communications for a given test of anIntermediate Node 200. - In one embodiment, the
Content Request 500 and theContent Response 600 are stored for each test of theIntermediate Nodes 200 in theLogical Data Repository 1200. In another embodiment, local tests are run on the Controlled RespondingNode 1450 and the results of the local tests are combined with theContent Request 500 andContent Response 600. Any combination of this data is referred to as Responding Node Data 1900 for the purposes of this discussion. - Depending on the requirements of a given embodiment, the Controlled Requesting
Node 1300 may iterate through a series of tests to determine desired functionality and safety-related data of theIntermediate Node 200. For example, different types of communication may be attempted by the Controlled Requesting Node BOO-including, for example, using different types of communication standards (i.e. HTTP 1.0vs. HTTP 1.1), different TCP commands (Put, Get, Post) and different HTTP/TCP Header values. These different types of communication attempts, along with the various types ofTarget Content 300 being retrieved, enable the IntermediateNode Verification System 1500 to determine the functionality available for a givenIntermediate Node 200. For example, if aNode Request 400 using an HTTP 1.1 GET commands for streaming media returns valid results, the capabilities of thisIntermediate Node 200 required for returning such results are discernible. - According to another embodiment, the Controlled Requesting
Node 1300 includes static values in theNode Request 400 from the Controlled RequestingNode 1300 that are optionally checked or utilized in some manner at the Controlled RespondingNode 1450. A further embodiment of theNode Verification System 1500 returns the static values via theContent Response 600 and/or adds in new static values into theContent Response 600. For example, the Controlled RequestingNode 1300 might include a static identifier in aNode Request 400. The static identifier may be a globally unique identifier, a checksum value of the content being sent or some other value therein. The value itself may be included in any part of theNode Request 400 including, but not limited to, the TCP Header, HTTP Header or TCP/HTTP message, or it may be transmitted via a separate channel depending on the embodiment. The Controlled RespondingNode 1450 may look for this value as a check on content safety and may, in turn, return a related identifier or new static values. The Controlled RequestingNode 1300 may look for the related identifier or new static values as a check on content safety. - In one embodiment, the Controlled Responding
Node 1450 adds scripts, values or pieces of code to determine what theIntermediate Node 200 is filtering during transmission. For example, if a client-side script is statically added, the Controlled RequestingNode 1300 can look for this script in theNode Request 400. If the script is missing, theNode Verification System 1500 may determine that theIntermediate Node 200 does not support scripting. Static values can be determined using configuration settings, be generated by an end user terminal directly in theLogical Data Repository 1200, or as part of an end user program running on the Controlled RequestingNode 1300 or result from rules applied against the Candidate Node Data 1700. - Further embodiments of the
Node Verification System 1500 performmultiple Node Requests 400 serially or in parallel to determine if theIntermediate Node 200 changes behavior from request to request or to determine the amount of traffic a givenIntermediate Node 200 can support. In another embodiment, theNode Verification System 1500 requests different types of data including text data or binary data or combinations thereof to determine whether theIntermediate Node 200 supports such data. - In a further embodiment, the
Node Verification System 1500 utilizes a known set of data sent from the Controlled RespondingNode 1450 to the Controlled RequestingNode 1300 to discern extra content inserted by theIntermediate Node 200. For example, if a givenIntermediate Node 200 embeds the data from the Controlled RespondingNode 1450 in a given TCP package field or nests the data in a given HTML element, theNode Verification System 1500 can utilize manual text inspection, Regex Expressions, workflow processes or other similar techniques to discern how the data from the Controlled RespondingNode 1450 is being included in theoverall Content Response 600 from theIntermediate Node 200. - In an another embodiment, being able to discern how the data from the Controlled Responding
Node 1450 is encapsulated in a givenContent Response 600 can enable the safe use of any otherwise unsafeIntermediate Node 200. In an another embodiment, theNode Verification System 1500 can either capture the static versions of data inserted by theIntermediate Node 200 or theNode Verification System 1500 can capture basic framing elements that wrap inserted data. As an example, if an HTTP Header packet contains specific values from theIntermediate Node 200, those values can be statically obtained and stored in places such as theLogical Data Repository 1200. As another possible example, if theNode Verification System 1500 discerns that extra HTML was inserted into aContent Response 600, the system might look for framing elements such as, but not limited to, table, body, div, span, p, Ii or input tags. By capturing these types of tags that sit at the beginning and end of inserted content, subsequent processes might be able to strip out the inserted content regardless of the dynamic nature of this inserted content. - The use of this optionally extracted data is not part of the
Node Verification System 1500 in this embodiment. Rather this embodiment provides an example of how data from the Controlled RequestingNode 1450 can be used to discern inserted content and optionally store said data in places such as theLogical Data Repository 1500. That data can then, in other embodiments, be used to safely interact withIntermediate Nodes 200 that would be otherwise unsafe to use or would fail business rules such as, for example, preventing ads from appearing on a user's browser. - C. Node Verification Process
- An Intermediate Node Verification Process begins with the Controlled Requesting
Node 1300 sending aNode Request 400 to Controlled RespondingNode 1450 based on the Candidate Node Data 1700 of theIntermediate Node 200 to be tested. TheIntermediate Node 200, if functional, receives theNode Request 400 and sends aContent Request 500 to the Controlled RespondingNode 1450. The Controlled RespondingNode 1450 optionally stores theContent Request 500 and then sends a knownContent Response 600 back to theIntermediate Node 200 candidate. The Controlled RespondingNode 1450 may send the Responding Node Data 1900 to theLogical Data Repository 1200. If the Candidate Node Data 1700 points to a node that is not anIntermediate Node 200, theNode Request 400 will fail and the node will not be categorized as anIntermediate Node 200. Alternative example embodiments of theNode Verification System 1500 may run additional tests to determine functionality of the nodes determined as non-Intermediate Nodes. - In an alternative embodiment, the Controlled Responding
Node 1450 createscontextual Content Responses 600, which include content related to the incoming request, and stores nothing. In another embodiment, the Controlled RequestingNode 1300 and Controlled RespondingNode 1450 directly communicate some combination of Requesting Node Data 1800 and Responding Node Data 1900 through any available communication type or path. In another embodiment, the Controlled RequestingNode 1300 sends outNode Requests 400 but stores no information. And in another embodiment, in which node connectivity and throughput are tested, the content of the requests and responses from the Controlled RequestingNode 1300, theIntermediate Node 200 and the Controlled RespondingNode 1450 are ignored and, instead transmission times, amongst other options are utilized to determine functionality and characteristics of theIntermediate Node 200. - If the
Intermediate Node 200 is a viable intermediate node, theIntermediate Node 200 sends anIntermediate Node Response 700 back to the Controlled RequestingNode 1300. In one embodiment, this process is optionally repeated across different communication types, paths and modes as well asTarget Content 300 types and optionally repeated more than one time. - Each
Intermediate Node Response 700 message may be stored and inspected for static values being sent from the Controlled RespondingNode 1450.Intermediate Node Responses 700 not matching knownContent Responses 600 indicate that the content returned from theIntermediate Node 200 is being modified by theIntermediate Node 200. Various checks for known scripts, tracking mechanisms and additional header values may be performed, and the system may use the testedIntermediate Nodes 200 despite the modifications depending on a type of the system. - For example, the Intermediate
Node Verification System 1500 may utilize checksum values to determine if network communications are being tampered with by theIntermediate Node 200. In a representative embodiment, the Controlled RequestingNode 1300 sends out astatic Node Request 400 through anIntermediate Node 200 to a Controlled RespondingNode 1450. Because theIntermediate Node 200 can be any network device including, but not limited to, routers, repeaters and bridges, this configuration utilizing checksum values is ideal for identifying corrupted messages indicative of network tampering. The Controlled RespondingNode 1450 performs a checksum on the receivedContent Request 500 and compares that value to an optionally encrypted checksum value sent from the Controlled RequestingNode 1300 either in the same message or as a separate, direct communication or even an indirect sharing of data as previously described. If the checksum values are different, the message is determined to have been modified. The same process can be performed and occur in the reverse with messages going from the Controlled RespondingNode 1450 to the Controlled RequestingNode 1300, and the two nodes can switch position such that the requesting node becomes the responding node and vice versa. - In another embodiment, the
Content Response 600 might be one of a plurality of possible responses that are optionally stored on both the Controlled RequestingNode 1300 and the Controlled RespondingNode 1450. In a further embodiment, the index or unique identifier for aspecific Content Response 600, optionally encrypted, can be included in theContent Response 600. In another embodiment, the two nodes might have an independent direct communications channel separate from theIntermediate Node 200 through which identifiers, checksums,Content Responses 600 or other information can be exchanged sufficient for the RequestingNode 1300 to discern whatContent Response 600 was sent to theIntermediate Node 200. Utilizing a plurality of possibleContent Responses 600 enables optional dynamic content checks,Intermediate Node 200 caching issues and optionally helps to obfuscate testing patterns. - According to one embodiment of the
Node Verification System 1500, any modifications to theContent Responses 600 may be sufficient to mark theIntermediate Node 200 as dangerous or nonviable and to exclude saidIntermediate Node 200 from subsequent usage. Other example embodiments may run further tests to determine the extent of modifications and whether said modifications occur across types, paths and modes of communication, before marking theIntermediate Node 200 as dangerous or nonviable. Further embodiments, as described previously, might look for mechanisms to enable safe usage of otherwiseunsafe Intermediate Nodes 200. - In one embodiment, upon completion of the various request cycles, the Controlled Requesting
Node 1300 may then inspect the Content Requests 500 made by theIntermediate Node 200. In other example embodiments, the inspection is performed by the Controlled RespondingNode 1450, by another application, program or system outside of the logical nodes in the IntermediateNode Verification System 1500 or some combination thereof wherein communications are utilized to distribute processes across internal and external nodes. - In an example embodiment, inspecting various TCP and/or HTTP fields, which are evident to any person of sufficient technical skill, may aid in determining the Intermediate Node's 200 presence, the Controlled Requesting Node's 1450 identity and the location from which Content Requests 500 were sent. By comparing these values to the values of the
other Content Requests 500, in an example further embodiment, the Node Verification Process determines if a singleIntermediate Node 200 is a conduit to more than oneIntermediate Node 200 on the outbound side. Further example embodiments of theNode Verification System 1500 determine differences in requests and identity information to determine the type ofIntermediate Node 200 and privacy level thereof. For example, someIntermediate Nodes 200 share the Requesting Node's 100 information while others completely hide such information. - In further embodiments, test cycles are repeated to determine changes in
Intermediate Node 200 functionality over time. Still further embodiments additionally repeat tests across communication types, paths and modes to categorize a range ofIntermediate Node 200 functionality. For example, a givenIntermediate Node 200 may successfully transmit HTML-based content but fail to transmit audio or video content. In such a case, theIntermediate Node 200 may be considered to support standard HTML content but not support multimedia content. Other example embodiments will continuously perform tests, or perform tests at periodic intervals to maintain current data on eachIntermediate Node 200. - By utilizing the information on both sides of the
Intermediate Node 200, the IntermediateNode Verification System 1500 is able to determine what theIntermediate Node 200 is actually doing as opposed to what it states it is doing, thereby enabling systems and methods according to example embodiments to identify and properly categorize each type ofIntermediate Node 200 along with its list of capabilities. -
FIG. 6 provides further logical node constructs in theNode Crawler System 1100 and the Intermediate Node Verification System 1500 (seeFIG. 2 ). In this embodiment, theNode Crawler 1100 is actually comprised of n-number physical systems that each run an independent crawling process or processes. In an embodiment, theseNode Crawlers 1100 store any found results in a local logical repository and then that repository's data is retrieved by theLogical Data Repository 1220. In another embodiment, theNode Crawlers 1100 send their collected data directly to the Logical Data Repository but can still optionally store some or all of their collected data in local logical data repositories. In a further embodiment there can exist a mix of Node Crawlers such that some utilize local logical data repositories and others can send data directly toother Crawler Nodes 1100 or theLogical Data Repository 1220. - As a further embodiment, the
Node Crawlers 1100 can further use either rules-based processing as described previously, external libraries or components or some combination therein to extract values from the incomingSource Node Data 1650. Depending on the specific embodiment,Node Crawlers 1100 might process the incoming data as it is received, they may cache the data for later processing or they may utilize a hybrid approach wherein data is processed as it is received unless the server load is to great—at which time the data is cached for later parsing. Further, the Candidate Node Data 1700 extraction process might be relegated to theLogical Data Repository 1220 exclusively or through a distributed process wherein certain components run on theNode Crawler 1100, such as external library calls, and other components, such as workflow processing, occur on theLogical Data Repository 1220 Node. - In a possible embodiment, with reference to
FIG. 2 and as shown inFIG. 6 , theLogical Data Repository 1220 Node is a physical device that accepts in Candidate Node Data 1700 and can send out data to theCrawler Nodes 1100. This latter process, in a further set of optional embodiments might be used to change global configuration settings in aspecific Node Crawler 1100 including, but not limited to, processor usage, memory consumption and network bandwidth limits. These changes might be direct modifications or indirect through mechanism such as the degree of parallelism allowed in a givenNode Crawler 1100. Further theLogical Data Repository 1220, depending on the embodiment, might update locally-stored data on a givenNode Crawler 100 which optionally modifies the behavior of thatNode Crawler 1100. - As an illustrative example, the Logical Data Repository might be configured such that the main domain of a website is stored in a parent table called Domains which has a child table called Pages. The Pages table, in turn, could optionally be a parent to a child table called Results. In the Results table, in an embodiment, bitwise flags could be utilized to determine if a given result in a valid
Intermediate Node 200 or not. Subsequent processing, depending on the embodiment, might then analyze the good and bad Results and determine that a given page in the Pages table is not producing any valid results. For example, a process might determine that a given page generated 10,000 results over a certain period of time but that none of those Results were valid. If that period of time crosses a threshold of allowable time to produce valid Candidate Node Data 1700 then the page itself may optionally be marked as Bad. - A further embodiment might then utilize a process to look at every Page within a Domain and see if any of the pages remain valid. In an optional embodiment, the inability to produce a valid page over some configurable amount of time might lead the system to mark the Domain as invalid/do not crawl. This process might run on each
specific Node Crawler 1100 locally, it might run as a system across allNode Crawlers 1100 but still distributed just on those crawlers; it might run directly on theLogical Data Repository 1220 with the data from theLogical Data Repository 1220 directly and being polled and updated directly on theNode Crawlers 100. Further, an optional embodiment might utilize an independent system, service, application or such mechanism running completely independently to determine these values. As shown inFIG. 6 , this type of processing may occur in the Management Server 3200. - The
Logical Data Repository 1220, in a given embodiment, might then provide data to a range of nodes as shown inFIG. 6 . In a given embodiment, the Management Server 3200 might monitor the data in theLogical Data Repository 1220 in order to determine when to launch actions such as data migration between nodes, setting flags as described previously or alerting other nodes to start their data-related processes. TheLogical Data Repository 1220 might share all or part of its stored data with theLogical Data Repository 1210. This shows the demarcation between a physical implementation and the logical node construct. The two nodes might collectively encapsulate all of theLogical Data Repository 1200 or they might be part of a distributed system that includes database servers found on theNode Crawlers 1100, theDiscovery Server 3300's database as well as theValidation Server 3000's local data repository. - While the
Discovery Server 3300 hosts the IntermediateNode Verification System 1500 in a specific embodiment,FIG. 6 actually shows how this logical process can be implemented and extended across two physical devices. TheDiscovery Server 3300, in accordance with this example embodiment, would provide the first test and determine the viability of a given URI as anIntermediate Node 200. TheValidation Server 3000, on the other hand, might perform a different level of inspection but perform that level on a regular or even continuous basis. This latter processing might perform a less intensive checking process or it might use a combination of the full IntermediateNode Verification System 1500's process with a lighter verification approach in response to speed and load demands. By dividing the logicalNode Verification System 1500 process across multiple physical devices, theIntermediate Nodes 200 can be initially verified and then regularly, irregularly or continuously verified to overcome modifications over time. - Thus, while there have been shown and described and pointed out fundamental novel features of the present disclosure as applied to preferred embodiments thereof, it will be understood that various omissions and substitutions and changes in the form and details of the devices described and illustrated, and in their operation, and of the methods described may be made by those skilled in the art without departing from the spirit of the present disclosure. For example, it is expressly intended that all combinations of those elements and/or method steps which perform substantially the same function in substantially the same way to achieve the same results are within the scope of the disclosure. Substitutions of elements from one described embodiment to another are also fully intended and contemplated. It is the intention, therefore, to be limited only as indicated by the scope of the claims appended hereto.
Claims (20)
1. A system for obtaining node data, the system comprising: at least one system node configured to obtain source node data of at least one source node, to apply at least one rule to the obtained source node data to extract candidate node data from the obtained source node data, and to send the candidate node data to a logical data repository.
2. The system of claim 1 , wherein the at least one system node comprises the logical data repository which stores the candidate node data.
3. The system of claim 1 , further comprising at least one other node external to the at least one system node, the at least one other node comprising the logical data repository which stores the candidate node data.
4. The system of claim 1 , wherein the at least one source node is a plurality of source nodes.
5. The system of claim 4 , further comprising a set of source node information including at least one Uniform Resource Identifier (URI), wherein the at least one URI points to a corresponding source node of the plurality of source nodes, and the at least one system node obtains the source node data for the corresponding source node based on the at least one URI.
6. The system of claim 1 , wherein the at least one rule comprises a plurality of rules that are selected from the group consisting of: regular expressions, publically available processing sources, WebKit sources and combinations of any of the foregoing.
7. The system of claim 6 , wherein the at least one system node iteratively applies at least a portion of the plurality of rules to the source node data of the at least one node to extract the candidate node data from the source node data.
8. The system of claim 6 , wherein the plurality of rules are configured to extract candidate node data having a plurality of different formats.
9. The system of claim 6 , wherein the plurality of rules comprise at least one regular expression command or at least one custom expression command.
10. The system of claim 9 , wherein the at least one custom expression command comprises a workflow process which is iteratively applied to the source node data of the at least one source node.
11. The system of claim 6 , further comprising a rule generation tool configured to create rules based on input from a user and store the created rules in the dynamic rule repository.
12. The system of claim 6 , wherein, after a first rule is applied which successfully obtains candidate node data from the source node data of the at least one source node, the at least one system node does not apply the remainder of the plurality of rules to the source node data of the at least one source node.
13. The system of claim 6 , wherein the at least one system node applies the plurality of rules by successively iterating through the plurality of rules in increasing complex logic to progressively filter content from the source node data until a desired level of the candidate node data is obtained.
14. The system of claim 1 , wherein the at least one source node is a plurality of source nodes of a given network, and the at least one system node is configured to apply the at least one rule to the source node data of each of the plurality of source nodes of the given network to extract to extract candidate node data for each said node of the given network, and send the candidate node data for each said node of the given network to the logical data repository.
15. A method for obtaining and storing node data, the method comprising:
obtaining, by at least one system node, source node data of at least one source node;
applying, by the at least one system node, at least one rule to the source node data to extract candidate node data from the source node data; and
sending, by the at least one node, the candidate node data to a logical data repository.
16. The method of claim 15 , further comprising storing, by the logical data repository, the candidate node data in the at least one system node.
17. The method of claim 15 , further comprising storing, by the logical data repository, the candidate node data in at least one other node external to the at least one system node.
18. The method of claim 15 , wherein the at least one source node is a plurality of source nodes.
19. The method of claim 18 , further comprising reading, by the at least one system node, a set of source node information including at least one Uniform Resource Identifier (URI), wherein the at least one URI points to a corresponding source node of the plurality of source nodes, and the source node data for the corresponding source node is obtained by the at least one system node based on the at least one URI.
20. The method of claim 15 , wherein the at least one rule is a plurality of rules stored by a dynamic rule repository.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/162,255 US20160269248A1 (en) | 2012-05-07 | 2016-05-23 | Systems and methods for detecting, identifying and categorizing intermediate nodes |
US16/152,102 US20190104025A1 (en) | 2012-05-07 | 2018-10-04 | Systems and methods for detecting, identifying and categorizing intermediate nodes |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/465,799 US9348927B2 (en) | 2012-05-07 | 2012-05-07 | Systems and methods for detecting, identifying and categorizing intermediate nodes |
US15/162,255 US20160269248A1 (en) | 2012-05-07 | 2016-05-23 | Systems and methods for detecting, identifying and categorizing intermediate nodes |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/465,799 Continuation US9348927B2 (en) | 2012-05-07 | 2012-05-07 | Systems and methods for detecting, identifying and categorizing intermediate nodes |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/152,102 Continuation US20190104025A1 (en) | 2012-05-07 | 2018-10-04 | Systems and methods for detecting, identifying and categorizing intermediate nodes |
Publications (1)
Publication Number | Publication Date |
---|---|
US20160269248A1 true US20160269248A1 (en) | 2016-09-15 |
Family
ID=49513436
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/465,799 Active US9348927B2 (en) | 2012-05-07 | 2012-05-07 | Systems and methods for detecting, identifying and categorizing intermediate nodes |
US15/162,255 Abandoned US20160269248A1 (en) | 2012-05-07 | 2016-05-23 | Systems and methods for detecting, identifying and categorizing intermediate nodes |
US16/152,102 Abandoned US20190104025A1 (en) | 2012-05-07 | 2018-10-04 | Systems and methods for detecting, identifying and categorizing intermediate nodes |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/465,799 Active US9348927B2 (en) | 2012-05-07 | 2012-05-07 | Systems and methods for detecting, identifying and categorizing intermediate nodes |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/152,102 Abandoned US20190104025A1 (en) | 2012-05-07 | 2018-10-04 | Systems and methods for detecting, identifying and categorizing intermediate nodes |
Country Status (3)
Country | Link |
---|---|
US (3) | US9348927B2 (en) |
EP (1) | EP2847696B1 (en) |
WO (1) | WO2013169685A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9992180B2 (en) | 2012-05-24 | 2018-06-05 | Smart Security Systems Llc | Systems and methods for protecting communications between nodes |
US10382595B2 (en) | 2014-01-29 | 2019-08-13 | Smart Security Systems Llc | Systems and methods for protecting communications |
US10778659B2 (en) | 2012-05-24 | 2020-09-15 | Smart Security Systems Llc | System and method for protecting communications |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10049033B2 (en) * | 2014-06-03 | 2018-08-14 | Sap Se | Application gateway for cloud computing systems |
CN106603516B (en) * | 2016-12-02 | 2021-04-30 | 中科星图股份有限公司 | Data inspection method and system |
US11194930B2 (en) | 2018-04-27 | 2021-12-07 | Datatrendz, Llc | Unobtrusive systems and methods for collecting, processing and securing information transmitted over a network |
WO2020086605A1 (en) * | 2018-10-22 | 2020-04-30 | Affirmed Networks, Inc. | Distributed database-driven resource management and locking in a cloud native mobile core network node architecture |
US20220200973A1 (en) * | 2019-04-15 | 2022-06-23 | Bear System, LLC | Blockchain schema for secure data transmission |
US11483143B2 (en) * | 2019-04-15 | 2022-10-25 | Smart Security Systems, Llc | Enhanced monitoring and protection of enterprise data |
Citations (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020024931A1 (en) * | 2000-08-31 | 2002-02-28 | Tsutomu Chikazawa | Transmission apparatus with a function to switch a line in the event of a transmission failure |
US20030101253A1 (en) * | 2001-11-29 | 2003-05-29 | Takayuki Saito | Method and system for distributing data in a network |
US20030154448A1 (en) * | 2002-01-31 | 2003-08-14 | Steven Teig | Method and apparatus for producing a circuit description of a design |
US6690678B1 (en) * | 1998-11-10 | 2004-02-10 | International Business Machines Corporation | Method and system in a packet switching network for dynamically adjusting the bandwidth of a continuous bit rate virtual path connection according to the network load |
US20060293969A1 (en) * | 2005-06-28 | 2006-12-28 | Sean Barger | Method and System for Pre-Loading Media Players |
US20070011146A1 (en) * | 2000-11-15 | 2007-01-11 | Holbrook David M | Apparatus and methods for organizing and/or presenting data |
US20070150075A1 (en) * | 2005-09-02 | 2007-06-28 | Dumas Marlon G | Process model transformation for event-based coordination of composite applications |
US20070165625A1 (en) * | 2005-12-01 | 2007-07-19 | Firestar Software, Inc. | System and method for exchanging information among exchange applications |
US20070253427A1 (en) * | 2006-04-28 | 2007-11-01 | Nokia Corporation | System, Method and Computer Program Product for Providing Quality of Service During Remote Access To a Plug-and-Play Network |
US20070291780A1 (en) * | 2006-06-16 | 2007-12-20 | Harris Corporation | System and methods for generic data transparent rules to support quality of service |
US20070300290A1 (en) * | 2002-11-18 | 2007-12-27 | Trusted Network Technologies | Establishing Secure TCP/IP Communications Using Embedded IDs |
US20080279198A1 (en) * | 2003-10-31 | 2008-11-13 | Ingo Gruber | Method for the Transmission of Information in a Communication System Using a Path |
US20090070771A1 (en) * | 2007-08-31 | 2009-03-12 | Tom Silangan Yuyitung | Method and system for evaluating virtualized environments |
US20090319661A1 (en) * | 2008-06-24 | 2009-12-24 | Fujitsu Limited | Cluster node control apparatus of file server |
US20110243024A1 (en) * | 2008-12-02 | 2011-10-06 | Oesterling Jacob | Method and apparatus for influencing the selection of peer data sources in a p2p network |
US20120011098A1 (en) * | 2009-03-19 | 2012-01-12 | Murakumo Corporation | Method and system for managing replication of data |
US20120246126A1 (en) * | 2003-10-27 | 2012-09-27 | Archivas, Inc. | Policy-based management of a redundant array of independent nodes |
US20130232263A1 (en) * | 2009-12-18 | 2013-09-05 | Morningside Analytics | System and method for classifying a contagious phenomenon propagating on a network |
US20130297703A1 (en) * | 2011-01-14 | 2013-11-07 | Alcatel-Lucent | Peer node and method for improved peer node selection |
US20150081868A1 (en) * | 2006-04-21 | 2015-03-19 | Cirba Inc. | Method and system for determining compatibility of computer systems |
US20150172108A1 (en) * | 2008-11-27 | 2015-06-18 | Huawei Device Co., Ltd. | Apparatus and method for locating a target operation object |
US9391832B1 (en) * | 2011-12-05 | 2016-07-12 | Menlo Security, Inc. | Secure surrogate cloud browsing |
Family Cites Families (44)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6212574B1 (en) | 1997-04-04 | 2001-04-03 | Microsoft Corporation | User mode proxy of kernel mode operations in a computer operating system |
US6131163A (en) | 1998-02-17 | 2000-10-10 | Cisco Technology, Inc. | Network gateway mechanism having a protocol stack proxy |
US6233618B1 (en) | 1998-03-31 | 2001-05-15 | Content Advisor, Inc. | Access control of networked data |
USH2065H1 (en) | 1998-12-28 | 2003-05-06 | Multi-Tech Systems, Inc. | Proxy server |
US6298383B1 (en) | 1999-01-04 | 2001-10-02 | Cisco Technology, Inc. | Integration of authentication authorization and accounting service and proxy service |
US6408061B1 (en) | 1999-03-12 | 2002-06-18 | Nortel Networks Limited | Interfacing a communication switch to a non-embedded device driver via a virtual device interface |
US7954144B1 (en) | 2000-01-18 | 2011-05-31 | Novell, Inc. | Brokering state information and identity among user agents, origin servers, and proxies |
US7003571B1 (en) | 2000-01-31 | 2006-02-21 | Telecommunication Systems Corporation Of Maryland | System and method for re-directing requests from browsers for communication over non-IP based networks |
US6754709B1 (en) | 2000-03-29 | 2004-06-22 | Microsoft Corporation | Application programming interface and generalized network address translator for intelligent transparent application gateway processes |
US6721779B1 (en) | 2000-07-07 | 2004-04-13 | Softwired Ag | Messaging proxy system |
US7461150B1 (en) | 2000-07-19 | 2008-12-02 | International Business Machines Corporation | Technique for sending TCP messages through HTTP systems |
US7417568B2 (en) | 2000-10-03 | 2008-08-26 | Realtime Data Llc | System and method for data feed acceleration and encryption |
US20020133709A1 (en) | 2001-03-14 | 2002-09-19 | Hoffman Terry George | Optical data transfer system - ODTS; Optically based anti-virus protection system - OBAPS |
US6850986B1 (en) | 2001-03-21 | 2005-02-01 | Palm Source, Inc. | Method and system for implementing URL scheme proxies on a computer system |
US8020201B2 (en) | 2001-10-23 | 2011-09-13 | Intel Corporation | Selecting a security format conversion for wired and wireless devices |
US7958550B2 (en) | 2001-11-02 | 2011-06-07 | Sterling Commerce, Inc. | Method and system for secure communication |
US20050033988A1 (en) | 2002-10-18 | 2005-02-10 | Neoscale Systems, Inc. | Method and system for transparent encryption and authentication of file data protocols over internet protocol |
US7467399B2 (en) * | 2004-03-31 | 2008-12-16 | International Business Machines Corporation | Context-sensitive confidentiality within federated environments |
US7966654B2 (en) | 2005-11-22 | 2011-06-21 | Fortinet, Inc. | Computerized system and method for policy-based content filtering |
WO2008004064A1 (en) | 2006-06-30 | 2008-01-10 | Network Box Corporation Limited | Proxy server |
US7865717B2 (en) * | 2006-07-18 | 2011-01-04 | Motorola, Inc. | Method and apparatus for dynamic, seamless security in communication protocols |
WO2008111051A2 (en) | 2007-03-09 | 2008-09-18 | Ghost, Inc. | A general object graph for web users |
US8189512B2 (en) | 2007-03-23 | 2012-05-29 | Telefonaktibolaget L M Ericsson (Publ) | Proxy mobile IP routing |
JP4981544B2 (en) | 2007-06-27 | 2012-07-25 | 富士フイルム株式会社 | Communication system, proxy server, control method thereof, and control program thereof |
US8019884B2 (en) | 2007-12-27 | 2011-09-13 | International Business Machines Corporation | Proxy content for submitting web service data in the user's security context |
US8839403B2 (en) | 2007-12-31 | 2014-09-16 | Sandisk Il Ltd. | Local proxy system and method |
US9578123B2 (en) | 2008-01-08 | 2017-02-21 | International Business Machines Corporation | Light weight portal proxy |
US7664862B2 (en) | 2008-01-14 | 2010-02-16 | International Business Machines Corporation | Browser-based proxy server for customization and distribution of existing applications |
US20110016197A1 (en) | 2008-03-05 | 2011-01-20 | Yoshiko Shiimori | Proxy server, and method and program for controlling same |
JP2009284448A (en) * | 2008-05-26 | 2009-12-03 | Nippon Telegr & Teleph Corp <Ntt> | Method, system, and program for controlling overlay network communication path |
US8606967B2 (en) | 2008-06-17 | 2013-12-10 | Qualcomm Incorporated | Methods and apparatus for proxying of devices and services using overlay networks |
US8352729B2 (en) * | 2008-07-29 | 2013-01-08 | International Business Machines Corporation | Secure application routing |
US8873578B2 (en) | 2008-11-12 | 2014-10-28 | Telefonaktiebolaget L M Ericsson (Publ) | Method and apparatus for use in a communications network |
US20100153568A1 (en) | 2008-12-16 | 2010-06-17 | Nokia Corporation | Methods, apparatuses, and computer program products for providing a local proxy for accessing web services |
US8732700B2 (en) | 2008-12-18 | 2014-05-20 | Vmware, Inc. | Virtualization system with a remote proxy |
US20100174817A1 (en) | 2009-01-06 | 2010-07-08 | Chetuparambil Madhu K | Splicing proxied web requests with callback for subsequent requests |
US20100175122A1 (en) | 2009-01-08 | 2010-07-08 | Verizon Corporate Resources Group Llc | System and method for preventing header spoofing |
US20100205215A1 (en) | 2009-02-11 | 2010-08-12 | Cook Robert W | Systems and methods for enforcing policies to block search engine queries for web-based proxy sites |
US9003429B2 (en) | 2009-09-23 | 2015-04-07 | Aliphcom | System and method of enabling additional functions or services of device by use of transparent gateway or proxy |
US8321502B2 (en) | 2010-03-02 | 2012-11-27 | Usablenet Inc. | Method for optimizing a web content proxy server and devices thereof |
US8700892B2 (en) | 2010-03-19 | 2014-04-15 | F5 Networks, Inc. | Proxy SSL authentication in split SSL for client-side proxy agent resources with content insertion |
US20110231479A1 (en) | 2010-03-22 | 2011-09-22 | Siemens Product Lifecycle Management Software Inc. | System and Method for Secure Multi-Client Communication Service |
US9634993B2 (en) | 2010-04-01 | 2017-04-25 | Cloudflare, Inc. | Internet-based proxy service to modify internet responses |
US8738656B2 (en) * | 2010-08-23 | 2014-05-27 | Hewlett-Packard Development Company, L.P. | Method and system for processing a group of resource identifiers |
-
2012
- 2012-05-07 US US13/465,799 patent/US9348927B2/en active Active
-
2013
- 2013-05-07 WO PCT/US2013/039805 patent/WO2013169685A1/en active Application Filing
- 2013-05-07 EP EP13788414.4A patent/EP2847696B1/en not_active Not-in-force
-
2016
- 2016-05-23 US US15/162,255 patent/US20160269248A1/en not_active Abandoned
-
2018
- 2018-10-04 US US16/152,102 patent/US20190104025A1/en not_active Abandoned
Patent Citations (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6690678B1 (en) * | 1998-11-10 | 2004-02-10 | International Business Machines Corporation | Method and system in a packet switching network for dynamically adjusting the bandwidth of a continuous bit rate virtual path connection according to the network load |
US20020024931A1 (en) * | 2000-08-31 | 2002-02-28 | Tsutomu Chikazawa | Transmission apparatus with a function to switch a line in the event of a transmission failure |
US20070011146A1 (en) * | 2000-11-15 | 2007-01-11 | Holbrook David M | Apparatus and methods for organizing and/or presenting data |
US20030101253A1 (en) * | 2001-11-29 | 2003-05-29 | Takayuki Saito | Method and system for distributing data in a network |
US20030154448A1 (en) * | 2002-01-31 | 2003-08-14 | Steven Teig | Method and apparatus for producing a circuit description of a design |
US20070300290A1 (en) * | 2002-11-18 | 2007-12-27 | Trusted Network Technologies | Establishing Secure TCP/IP Communications Using Embedded IDs |
US20120246126A1 (en) * | 2003-10-27 | 2012-09-27 | Archivas, Inc. | Policy-based management of a redundant array of independent nodes |
US20080279198A1 (en) * | 2003-10-31 | 2008-11-13 | Ingo Gruber | Method for the Transmission of Information in a Communication System Using a Path |
US20060293969A1 (en) * | 2005-06-28 | 2006-12-28 | Sean Barger | Method and System for Pre-Loading Media Players |
US20070150075A1 (en) * | 2005-09-02 | 2007-06-28 | Dumas Marlon G | Process model transformation for event-based coordination of composite applications |
US20070165625A1 (en) * | 2005-12-01 | 2007-07-19 | Firestar Software, Inc. | System and method for exchanging information among exchange applications |
US20150081868A1 (en) * | 2006-04-21 | 2015-03-19 | Cirba Inc. | Method and system for determining compatibility of computer systems |
US20070253427A1 (en) * | 2006-04-28 | 2007-11-01 | Nokia Corporation | System, Method and Computer Program Product for Providing Quality of Service During Remote Access To a Plug-and-Play Network |
US20070291780A1 (en) * | 2006-06-16 | 2007-12-20 | Harris Corporation | System and methods for generic data transparent rules to support quality of service |
US20090070771A1 (en) * | 2007-08-31 | 2009-03-12 | Tom Silangan Yuyitung | Method and system for evaluating virtualized environments |
US20090319661A1 (en) * | 2008-06-24 | 2009-12-24 | Fujitsu Limited | Cluster node control apparatus of file server |
US20150172108A1 (en) * | 2008-11-27 | 2015-06-18 | Huawei Device Co., Ltd. | Apparatus and method for locating a target operation object |
US20110243024A1 (en) * | 2008-12-02 | 2011-10-06 | Oesterling Jacob | Method and apparatus for influencing the selection of peer data sources in a p2p network |
US20120011098A1 (en) * | 2009-03-19 | 2012-01-12 | Murakumo Corporation | Method and system for managing replication of data |
US20130232263A1 (en) * | 2009-12-18 | 2013-09-05 | Morningside Analytics | System and method for classifying a contagious phenomenon propagating on a network |
US20130297703A1 (en) * | 2011-01-14 | 2013-11-07 | Alcatel-Lucent | Peer node and method for improved peer node selection |
US9391832B1 (en) * | 2011-12-05 | 2016-07-12 | Menlo Security, Inc. | Secure surrogate cloud browsing |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9992180B2 (en) | 2012-05-24 | 2018-06-05 | Smart Security Systems Llc | Systems and methods for protecting communications between nodes |
US10637839B2 (en) | 2012-05-24 | 2020-04-28 | Smart Security Systems Llc | Systems and methods for protecting communications between nodes |
US10778659B2 (en) | 2012-05-24 | 2020-09-15 | Smart Security Systems Llc | System and method for protecting communications |
US10382595B2 (en) | 2014-01-29 | 2019-08-13 | Smart Security Systems Llc | Systems and methods for protecting communications |
Also Published As
Publication number | Publication date |
---|---|
US20190104025A1 (en) | 2019-04-04 |
US20130297606A1 (en) | 2013-11-07 |
US9348927B2 (en) | 2016-05-24 |
WO2013169685A1 (en) | 2013-11-14 |
EP2847696A1 (en) | 2015-03-18 |
EP2847696A4 (en) | 2016-01-06 |
EP2847696B1 (en) | 2018-03-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20190104025A1 (en) | Systems and methods for detecting, identifying and categorizing intermediate nodes | |
US10334016B2 (en) | System and method for context specific website optimization | |
Bujlow et al. | A survey on web tracking: Mechanisms, implications, and defenses | |
US10262142B2 (en) | Systems and methods for advanced dynamic analysis scanning | |
US7647404B2 (en) | Method of authentication processing during a single sign on transaction via a content transform proxy service | |
AU2002252371B2 (en) | Application layer security method and system | |
US10887661B2 (en) | System and method for content monitoring and filtering to improve network efficiency | |
US11768898B2 (en) | Optimizing scraping requests through browsing profiles | |
US20240028721A1 (en) | Utilizing Machine Learning to detect malicious executable files efficiently and effectively | |
Atoum et al. | A hybrid technique for SQL injection attacks detection and prevention | |
Ashraf | Avoiding Vulnerabilities and Attacks with a Proactive Strategy for Web Applications | |
Kimak et al. | An investigation into possible attacks on HTML5 indexedDB and their prevention | |
Vilches et al. | Aztarna, a footprinting tool for robots | |
US20230353587A1 (en) | Contextual relationship graph based on user's network transaction patterns for investigating attacks | |
US20220067581A1 (en) | Utilizing Machine Learning for dynamic content classification of URL content | |
Yaacob et al. | Moving towards positive security model for web application firewall | |
Bodoarca et al. | Assuring Privacy in Surfing the Internet | |
Nyffenegger | Web for pentester | |
Athanasopoulos | Modern techniques for the detection and prevention of web2. 0 attacks | |
Acosta et al. | Automatic Traffic-Based Internet Control Message Protocol (ICMP) Model Generation for ns-3 | |
Sileika et al. | A Website Availability Check Script for Nagios | |
Brodar | Analysis of Exploit-kit Incidents and Cam-paigns Through a Graph Database Framework | |
GUSTAFSSON | Securing JavaScript applications within theSpotify web player | |
MUNCHEN | On detecting Web-Tracking |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SMART SECURITY SYSTEMS LLC, COLORADO Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:TOLA JR., KENNETH C.;REEL/FRAME:044822/0915 Effective date: 20180201 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |