Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L67/00—Network arrangements or protocols for supporting network services or applications
  • H04L67/01—Protocols
  • H04L67/10—Protocols in which an application is distributed across nodes in the network
  • H04L67/104—Peer-to-peer [P2P] networks
  • H04L67/1061—Peer-to-peer [P2P] networks using node-based peer discovery mechanisms
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/10—File systems; File servers
  • G06F16/18—File system types
  • G06F16/182—Distributed file systems
  • G06F16/1834—Distributed file systems implemented based on peer-to-peer networks, e.g. gnutella
  • G06F16/1837—Management specially adapted to peer-to-peer storage networks
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L63/00—Network architectures or network communication protocols for network security
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L67/00—Network arrangements or protocols for supporting network services or applications
  • H04L67/01—Protocols
  • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L67/00—Network arrangements or protocols for supporting network services or applications
  • H04L67/01—Protocols
  • H04L67/10—Protocols in which an application is distributed across nodes in the network
  • H04L67/104—Peer-to-peer [P2P] networks
  • H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
  • H04L67/1078—Resource delivery mechanisms
  • H04L67/108—Resource delivery mechanisms characterised by resources being split in blocks or fragments

Definitions

• the present disclosure relates generally to computer networks and more particularly to monitoring a BitTorrent network.
• the BitTorrent protocol is concerned with the reliable transfer of files. Users may search for torrent files, which are then used to download content of interest from
• a user may find particular torrent files by searching indexing sites/tracker sites by searching peer-to- peer (P2P networks, by joining Rich Site Summary (RSS) feeds, or by utilizing other types of BitTorrent management pages.
• P2P networks peer-to- peer networks
• RSS Rich Site Summary
• Each BitTorrent management page can have its own syntax and format. Accordingly, there is no single or consistent way to probe for context across the variety of different BitTorrent management we pages. Further, there is no centralized BitTorrent network where all participants can potentially be reached through connection hopping. Instead, each torrent "swarm" is an enclosed community, each Tracker site has no protocol level connection to the next, and indexing sites are disjoint from the one another.
• one or more BitTorrent management pages such as an index site or a Rich Site Summary (RSS) feed, may be scanned for indicia of a torrent file that is associated with one or more search terms.
• information associated with the torrent file may be utilized to initiate the downloading of one or more portions of the content indicated by the torrent file over a network from swarm computers utilizing a BitTorrent Protocol.
• identification information associated with each swarm computer may be obtained.
• Data associated with the torrent file, content associated with the torrent file, and the identification information associated with each of the swarm computers may be stored in a database. Thereafter, a client may search the database, or a different storage structure, that stores the data associated with the torrent files, the content, and the identification information to locate one or more swarm computers that may be sharing, for example, unauthorized or illegal information.
• results obtained from the novel torrent monitoring system may be cross-compared with other systems (e.g., peer-to-peer network scans) using the same search terms to classify or categorize the combined results according to a "threat" level.
• other systems e.g., peer-to-peer network scans
• torrent files, their content, and the swarm computers that share content associated with the torrent files may be efficiently monitored.
• Fig. 1 illustrates an example computer network
• Fig. 2 illustrates an example torrent monitoring system that may be utilized computer network of Fig. 1 ;
• Fig. 3 illustrates an example simplified procedure for monitoring BitTorrent;
• Fig. 4 illustrates an example simplified procedure for searching for information associated with the torrent monitoring system of the current application
• Fig. 5 illustrates an example system for cross-comparing information obtained from the torrent monitoring system with information obtained from other systems using the same search terms
• Fig. 6 illustrates an example simplified procedure for cross-comparing information obtained from the novel torrent monitoring system with information obtained from other systems using the same search terms.
• Fig. 1 illustrates an example computer network 100 illustratively comprising client computers 105, torrent monitoring system 200, database 110, one or more swarm computers 115, and BitTorrent management pages 118 interconnected by communication links 120.
• client computers 105 client computers 105
• torrent monitoring system 200 database 110
• swarm computers 115 swarm computers 115
• BitTorrent management pages 118 interconnected by communication links 120.
• client computers torrent monitoring system, database, swarm computers and/or links may be used in the computer network, and that the view shown herein is for simplicity.
• Client computer 105 and swarm computers 115 may be any general purpose data processor, such as a personal computer or a workstation.
• Database 110 is a conventional structure that organizes a collection of data as known by those skilled in the art.
• Fig. 2 illustrates an example torrent monitoring system 200 that may be used with one or more embodiments described herein.
• the torrent monitoring system 200 may comprise a plurality of network interfaces 210, one or more data processors 220, and a memory 240 interconnected by a system bus 250.
• the network interfaces 210 contain the mechanical, electrical, and signaling circuitry for communicating data over physical links coupled to the network 100.
• the network interfaces may be configured to transmit and/or receive data using a variety of different communication protocols, including, inter alia, TCP/IP, UDP, ATM, synchronous optical networks (SONET), wireless protocols, Frame Relay, Ethernet, Fiber Distributed Data Interface (FDDI), etc.
• a physical network interface 210 may also be used to implement one or more virtual network interfaces, such as for Virtual Private Network (VPN) access, known to those skilled in the art.
• VPN Virtual Private Network
• the memory 240 comprises a plurality of locations that are addressable by the processor(s) 220 and the network interfaces 210 for storing software programs and data structures associated with the embodiments described herein.
• the processor 220 may comprise necessary elements or logic adapted to execute the software programs and manipulate the data structures.
• An operating system 242 portions of which are typically resident in memory 240 and executed by the processor(s), functionally organizes the node by, inter alia, invoking network operations in support of software processes and/or services executing on the node. These software processes and/or services may comprise scheduler, 238, scanner 244, torrent downloader 246, DSP matcher 250, and content downloader 252.
• Scheduler 238 is a process responsible for recurring execution of scanner 244, that is described below. Since indexing sites are a centralized repository and constant repeated probing can lead to the scheduler 238 being banned from accessing the indexing site, batch processing for torrent file discovery from a BitTorrent management page is advantageous. As such, scheduler 238 may minimize the footprint on the BitTorrent management page by batching the access to the BitTorrent management page, interleaving access across BitTorrent management pages to maximize time between access per site, and per site throttling.
• Each schedule defined by scheduler 238 may be with configuration items such as a recurrence value (e.g., every X hours, start/stop date), type of scan (RSS, search, walk, import, etc.) and a scan type input source value (e.g., for RSS: list of RSS feeds, for search: list of indexing sites, for walk: list of indexing sites, and for import: import protocol source).
• Scheduler 238 may be managed by an end user, for example an administrator using client 105.
• Scanner(s) 244 is a process responsible for discovering torrent files from a variety of BitTorrent "management page" types. These management pages provide indicia as to how to find torrent files, and may include indexing web sites, tracker sites, RSS feeds, etc. Each scanner 244 is configured to scan the BitTorrent management page according to the BitTorrent management page's syntax and/or format. For example, an indexing site may have a different syntax and/or format from an RSS feed, or may even have a different format than another indexing site. As such, scanners 244 are configured to interact with a variety of different BitTorrent management pages having different syntax and/or format.
• Each scanner 244 may receive a set of search terms of interest (referred to here as the digital signature profile (DSP) search terms) from a user, utilizing client 105 for example.
• DSP digital signature profile
• the DSP search terms are then used to scan the associated type of BitTorrent management pages for associated torrents, in a manner described in more detail below.
• Scanners 244 may include an "Index Scanner" (IS) that is designed to scan BitTorrent management pages that are of the indexing site type.
• the IS 244 may search indexing sites for torrents using the specific DSP search terms.
• the search- ability of the IS 244 is constrained by the indexing algorithm implemented and exposed by the indexing site. For example, more sophisticated indexing sites allow qualified search by popularity, by timeliness, by genre, etc, while less sophisticated indexing sites may only sort result by upload time, and lookup index of torrent file name. Other available indexing criteria may include torrent file name, content file name, descriptions and metadata.
• the IS 244 can form site specific URLs that contain the configured DSP terms. Each term, for example, may require one search request per indexing site.
• Scanners 244 may also include a "RSS Scanner" (RSSS) that is specifically purposed to discover available torrents published by RSS feeds.
• RSSS contacts the RSS syndication site(s) to download RSS feeds.
• RSS site format torrent URLs may be specified on the RSS XML itself, or indirectly on linked HTTP page(s).
• Scanners 244 may also include a "Walk Scanner" (WK) that is configured to scan for new torrents that were added to indexing sites since a previous walk. On each scan, the WK starts after the last torrent from a previous scan, then sequentially walk up to a last available torrent. Utilizing the WK has the advantage of not rediscovering a torrent more than once across walks, being exempt from result limits arbitrarily imposed by indexing sites, and sensitive search terms are never transmitted. The WK utilizes the sequential number of uploaded torrents and availability of "latest torrent" page provided by the indexing site.
• WK Walk Scanner
• Scanners 244 may also include an "Import Scanner" (ImS) that scans for torrent files made available via Peer-to-Peer (P2P) protocol engines.
• ImS Import Scanner
• P2P Peer-to-Peer
• the ImS assumes that the P2P protocol engines deliver files to a pre-determine import folder.
• An ImS periodically scans the import folder for new torrent file.
• IS can be configured to either truncate, or delete torrents at the data store maintained by the P2P protocol engines.
• Each type of scanner 244 (IS, RSSS, WK, ImS) thus scans is associated type of BitTorrent management page(s) and produces one or more torrent file indicators as an output.
• a torrent downloader 246 then uses this output and invokes a site specific JavaScript to extract the torrent download URL, or magnet links from the BitTorrent management page(s). If a torrent is unable be downloaded, or fails, a schedule may be created by specifying a recurrence interval to attempt to re-download the torrent.
• the torrent files are hosted by the BitTorrent management page(s), however, in other embodiments, the BitTorrent management pages(s) may store a key or "fingerprint" associated with a torrent file. Thereafter, the key or fingerprint may be utilized to obtain the torrent file from a decentralized network, for example a Distributed Hash Table (DHT) network.
• DHT Distributed Hash Table
• unsuccessfully downloaded torrents may be stored in a table of the database 110, or other data structure for re-download.
• Torrents are identified relative to the Uniform Resource Identifier (URI) associated with the BitTorrent management page on which they were discovered.
• URI Uniform Resource Identifier
• Uniqueness may be enforced in the database using the URI as a primary key.
• the torrent files themselves are also subject to indexing. Because torrent files are in binary "Bencode" format, they are not submitted directly to an indexer. Rather, a text file may be generated in which ancillary data collected during discovery of the torrent are included. The ancillary data yields additional information that is useful in forensic investigation and may include: index site detail page URL, index site torrent download URL, index site detail page description of the torrent, user who posted the torrent to the index site, RSS description for the torrent, etc. Successfully downloaded torrents stored in database 110 may be subsequently subject to a DSP matching/filtering process performed by DSP matcher 250.
• Successfully matched torrents may then be utilized by content downloader 252 to download content associated with the successfully matched torrents.
• Content downloader 252 continually scans for outstanding swarm jobs to execute that are queued after DSP matcher 250 performs its filtering.
• Content downloader 252 cycles through outstanding swarm jobs and attempts to connect to each swarm to download the associated content.
• Content downloader 252 utilizes the corresponding BitTorrent swarming protocol to discover swarm computers who are sharing (seeding) or downloading (leeching) from the swarm.
• content downloader 252 may utilize different trackers to identify the swarm computers that are either sharing or interested in downloading the content depending on the protocol specified by the torrent.
• Such trackers may include a Hypertext Transfer Protocol (HTTP) tracker, User Datagram Protocol (UDP) tracker, DHT tracker, Peer Exchange (PEX) Tracker and the like.
• HTML Hypertext Transfer Protocol
• UDP User Datagram Protocol
• DHT tracker Dynamic Host Transfer Protocol
• PEX Peer Exchange
• content downloader 252 may obtain identification information associated with the individual swarm computers that participate in the torrent. For example, once the swarm computer is identified, the torrent monitoring system 200 may establish a TCP connection with the swarm computer and start downloading the one or more portions of content according to the BitTorrent protocol. During establishment of the TCP connection, the torrent monitoring system 200 may learn the IP address, and other identification information associated with the swarm computer(s). The identification information, as well as the content, may then be stored at database 110. Further, content downloader 252 may reschedule those torrents without an active swarm.
• Fig. 3 illustrates an example simplified procedure for monitoring BitTorrent.
• the procedure 300 starts at step 305 and continues to step 310, where a client, for example client 105, selects one or more DSP search terms.
• the DSP search terms are received over the network 100 at torrent monitoring system 200.
• scanner 244 of torrent monitoring system 200 utilizes the received DSP search terms to scan one or more BitTorrent management pages 118 over network 100 to obtain indicia associated with the torrent files that those pages 118 manage. Since different BitTorrent management pages may have a different syntax and/or format, the scanning in step 320 invokes the appropriate type of scanner 244 (IS, RSSS, WK, ImS, etc.).
• torrent downloader 246 may extract or download one or more torrent files that are associated with the indicia discovered by scanner 244. For example, torrent downloader 246 may download the one or more torrent files from the BitTorrent management page(s) that host the torrent files. In a different embodiment, the torrent downloader 246 may utilize keys or fingerprints hosted by the BitTorrent management page(s) and associated with the torrent files to download the torrent files from
• the extracted torrent files and information associated with the torrent files may be stored in database 110.
• a text file may be generated in which ancillary data collected during discover of the torrent may be stored in database 110.
• DSP matcher 250 may subject the torrent files previously stored in database 110 to a matching/filtering process.
• step 340 the matching torrents are queued in a queue associated with content downloader 252, and content downloader 252 initiates download of content associated with a particular matching torrent from one or more identified swarm computers 115.
• step 345 the downloaded content and identification information associated with the one or more identified swarm computers 115 may be stored in database 110.
• Fig. 4 illustrates an example simplified procedure for searching for information associated with the torrent monitoring system 200.
• the procedure 400 starts at step 405 and continues to step 410, where a client, for example client 105, selects one or more DSP search terms.
• step 410 the DSP search terms are received by the torrent monitoring system 200.
• step 415 the torrent monitoring system 200 utilizes the received DSP search terms to search the database for matching torrent information and/or identification information associated with one or more swarm computers 115.
• step 420 the matching torrent information and/or identification information associated with the one or more swarm computers 115, may be sent to client 105. The procedure ends at step 425.
• the search term-related information e.g., torrent file information, identification information associated with swarm computers
• the novel torrent monitoring system 200 may be cross-compared against information obtained from other systems using the same search terms, such as Internet-based search engines or peer-to-peer network searching tools.
• Figure 5 is an example system 500 for cross-comparing information obtained from the torrent monitoring system 200 with information obtained from other systems 505 using the same search terms.
• Another search system is described in co-pending U.S. Patent Application Serial No. 13/706,703 filed on December 6, 2012 entitled SYSTEM FOR FORENSIC ANALYSIS OF SEARCH TERMS".
• each system can be configured to scan its respective information source (e.g., database 110) and provides results to filters 515 and 520 that may perform filtering operations based on file titles, file copies, etc.
• Other information such as IP addresses can provide further sort and/or matching, for example those found in database 110 associated with the one or more swarm computers.
• results from both systems may be stored in a centralized data storage system such as database 525 where intent and threats can be categorized (e.g., to determine if the scan pattern is indicative of information concentrators, hacker threats, physical/terror threats, or corporate security threats) by the capabilities of either the torrent monitoring system 200 or the traditional P2P network scan system 505.
• a centralized data storage system such as database 525 where intent and threats can be categorized (e.g., to determine if the scan pattern is indicative of information concentrators, hacker threats, physical/terror threats, or corporate security threats) by the capabilities of either the torrent monitoring system 200 or the traditional P2P network scan system 505.
• Fig. 6 illustrates an example simplified procedure for cross-comparing information obtained from the novel torrent monitoring system 200 with information obtained from other systems using the same search terms.
• the procedure 600 starts at step 605 and continues to step 610, where the results from a P2P network scan system 505 and the results from the scan from the torrent monitoring system 200 are respectively filtered by filters 515 and 520, for example, wherein the scans utilizes the same search terms.
• step 615 the filtered results from the two systems are combined and stored at a centralized data storage system.
• step 620 the combined results are categorized according to intent and threat.
• the procedure ends.
• the various "data processors” and “computers” described herein may each be implemented by a physical or virtual general purpose computer having a central processor, memory, disk or other mass storage, communication interface(s), input/output (I/O) device(s), and other peripherals.
• the general purpose computer is transformed into the processors and executes the processes described above, for example, by loading software instructions into the processor, and then causing execution of the instructions to carry out the functions described.
• such a computer may contain a system bus, where a bus is a set of hardware lines used for data transfer among the components of a computer or processing system.
• the bus or busses are essentially shared conduit(s) that connect different elements of the computer system (e.g., processor, disk storage, memory, input/output ports, network ports, etc.) that enables the transfer of information between the elements.
• One or more central processor units are attached to the system bus and provide for the execution of computer instructions.
• I/O device interfaces for connecting various input and output devices (e.g., keyboard, mouse, displays, printers, speakers, etc.) to the computer.
• Network interface(s) allow the computer to connect to various other devices attached to a network.
• Memory provides volatile storage for computer software instructions and data used to implement an embodiment.
• Disk or other mass storage provides non-volatile storage for computer software instructions and data used to implement, for example, the various procedures described herein.
• Embodiments may therefore typically be implemented in hardware, firmware, software, or any combination thereof.
• the computers that execute the processes described above may be deployed in a cloud computing arrangement that makes available one or more physical and/or virtual data processing machines via a convenient, on-demand network access model to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.
• configurable computing resources e.g., networks, servers, storage, applications, and services
• firmware, software, routines, or instructions may be described herein as performing certain actions and/or functions. However, it should be appreciated that such descriptions contained herein are merely for convenience and that such actions in fact result from computing devices, processors, controllers, or other devices executing the firmware, software, routines, instructions, etc.
• block and network diagrams may include more or fewer elements, be arranged differently, or be represented differently. But it further should be understood that certain implementations may dictate the block and network diagrams and the number of block and network diagrams illustrating the execution of the embodiments be implemented in a particular way.

Landscapes

• Engineering & Computer Science (AREA)
• Computer Networks & Wireless Communication (AREA)
• Signal Processing (AREA)
• General Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Computer Hardware Design (AREA)
• Computer Security & Cryptography (AREA)
• Computing Systems (AREA)
• Databases & Information Systems (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Data Mining & Analysis (AREA)
• Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
• Computer And Data Communications (AREA)
• Environmental & Geological Engineering (AREA)
• Information Transfer Between Computers (AREA)
• Data Exchanges In Wide-Area Networks (AREA)

Priority Applications (6)

Applications Claiming Priority (4)

Publications (1)

Family

ID=48670090

Family Applications (1)

Country Status (7)

Families Citing this family (7)

Family Cites Families (15)

• 2013
  • 2013-06-06 JP JP2015516188A patent/JP2015529868A/ja active Pending
  • 2013-06-06 EP EP13730423.4A patent/EP2859686B1/en active Active
  • 2013-06-06 AU AU2013271563A patent/AU2013271563B2/en active Active
  • 2013-06-06 CA CA2875524A patent/CA2875524C/en active Active
  • 2013-06-06 CN CN201380041866.0A patent/CN104521183B/zh active Active
  • 2013-06-06 WO PCT/US2013/044429 patent/WO2013184870A1/en not_active Ceased
  • 2013-06-06 US US13/911,563 patent/US9432273B2/en active Active
• 2016
  • 2016-07-25 US US15/218,218 patent/US20170034020A1/en not_active Abandoned
• 2018
  • 2018-09-14 JP JP2018172025A patent/JP6714661B2/ja active Active

Non-Patent Citations (3)

Also Published As

Similar Documents

Legal Events