WO2006071560A2 - Network packet capture distributed storage system - Google Patents
Network packet capture distributed storage system Download PDFInfo
- Publication number
- WO2006071560A2 WO2006071560A2 PCT/US2005/045566 US2005045566W WO2006071560A2 WO 2006071560 A2 WO2006071560 A2 WO 2006071560A2 US 2005045566 W US2005045566 W US 2005045566W WO 2006071560 A2 WO2006071560 A2 WO 2006071560A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- slot
- data
- volatile storage
- network
- buffer
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/02—Capturing of monitoring data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/561—Adding application-functional data or data for application control, e.g. adding metadata
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/06—Generation of reports
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0823—Errors, e.g. transmission errors
- H04L43/0829—Packet loss
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
- H04L43/106—Active monitoring, e.g. heartbeat, ping or trace-route using time related information in packets, e.g. by adding timestamps
Definitions
- the present invention relates to capturing and archiving computer network traffic.
- Networks allowing computer users to communicate and share information with one another are ubiquitous in business, government, educational institutions, and homes.
- Computers communicate with one another through small and large local area networks (LANs) that may be wireless or based on hard-wired technology such as Ethernet or fiber optics.
- LANs local area networks
- WANs wide area networks
- Most local networks have the ability to communicate with other networks through wide area networks (WANs).
- WANs wide area networks
- the interconnectivity of these various networks ultimately enables the sharing of information throughout the world via the Internet.
- other information sharing devices may interact with these networks, including cellular telephones, personal digital assistants (PDAs) and other devices whose functionality may be enhanced by communication with other persons, devices, or systems.
- PDAs personal digital assistants
- a robust network packet capture and archiving system must utilize the maximum capabilities of the latest hardware technologies and must also avoid the bottlenecks inherent in current technologies. Using multiple gigabit Ethernet connections, arrays of large hard disk drives, and software that by-passes traditional bottlenecks by more direct communication with the various devices, it is possible to achieve packet capture and archiving on a scale capable of handling the traffic of the largest networks.
- the present invention describes an Infinite Network Packet Capture System (INPCS).
- the INPCS is a high performance data capture recorder capable of capturing and archiving all network traffic present on a single network or multiple networks.
- the captured data is archived onto a scalable, infinite, disk based LRU (least recently used) caching system at multiple gigabit (Gb) line speeds.
- LRU least recently used
- Gb gigabit
- the INPCS has the ability to capture and stream to disk all network traffic on a gigabit Ethernet network and allows this stored data to be presented as a Virtual File System (VFS) to end users.
- VFS Virtual File System
- the file system facilitates security, forensics, compliance, analytics and network management applications.
- the INPCS also supports this capability via T1/T3 and other network topologies that utilize packet based encapsulation methods.
- the INPCS does not require the configuration of a protocol stack, such as TCIP/IP, on the network capture device. As a result, the INPCS remains "invisible” or passive and thus not detectable or addressable from network devices being captured. Being undetectable and unaddressable, INPCS enhances security and forensic reliability as it cannot be modified or "hacked" from external network devices or directly targeted for attack from other devices on the network. INPCS also provides a suite of tools and exposes the captured data in time sequenced playback, as a virtual network interface or virtual Ethernet device, a regenerated packet stream to external network segments and as a VFS file system that dynamically generates industry standard LIBPCAP (TCPDUMP) file formats. These formats allow the capture data to be imported into any currently available or custom applications that that support LIBPCAP formats. Analysis of captured data can be performed on a live network via INPCS while the device is actively capturing and archiving data.
- TCPDUMP industry standard LIBPCAP
- the INPCS platform is rack mountable device capable of supporting large arrays of RAID 0/RAID 5 disk storage with high performance Input/Output (I/O) system architectures. Storage of high- density network traffic is achieved by using copy-less Direct Memory Access (DMA).
- the INPCS device can sustain capture and storage rates of over 350 MB/s (megabytes per second).
- the device can be attached to Ethernet networks via, copper or fiber via either a SPAN port router configuration or via an optical splitter.
- the INPCS also supports the ability to merge multiple captured streams of data into a consolidated time indexed capture stream to support asymmetrically routed network traffic as well as other merged streams for external access, facilitating efficient network management, analysis, and forensic uses.
- the INPCS software may be independently used as a standalone software package compatible with existing Linux network interface drivers. This offering of the INPCS technology provides a lower performance metric than that available in the integrated hardware/software appliance but has the advantage of being portable across the large base of existing Linux supported network drivers.
- the standalone software package for INPCS provides all the same features and application support as available with the appliance offering above described, but does not provide the high performance disk I/O and copy-less Direct Memory Access (DMA) switch technology of the integrated appliance.
- DMA Direct Memory Access
- Captured network traffic can be exposed to external appliances and devices or appropriate applications running on the INPCS appliance utilizing three primary methods: a VFS file system exposing PCAP formatted files, a virtual network interface (Ethernet) device and through a regenerated stream of packets to external network segments feeding external appliances.
- the INPCS file system acts as an on-disk LRU (least recently used) cache and recycles the oldest captured data when the store fills and allows continuous capture to occur with the oldest data either being recycled and overwritten or transferred to external storage captured network traffic. This architecture allows for an infinite capture system. Captured packets at any given time in the on-disk store represents a view in time of all packets captured from the oldest packets to the newest.
- a system may be configured to allow a predetermined time window on all network traffic from a network of a predetermined traffic capacity.
- a predetermined traffic capacity For example a business, government entity, or university can configure an appliance with sufficient disk array storage to allow examination and analysis of all traffic during the prior 24 hours, 48 hours, or any other predetermined time frame.
- FIGURE 1 depicts the hardware configuration of the INPCS appliance
- FIGURE 2 depicts an INPCS 8 x 400 Appliance Chassis
- FIGURE 3 depicts the INPCS appliance in a switch port analyzer configuration
- FIGURE 4 depicts the INPCS appliance in an asymmetric routed configuration
- FIGURE 5 depicts in the INPCS appliance in an in-line optical splitter configuration
- FIGURE 6 depicts a typical menu tree for the DSMON utility
- FIGURE 7 depicts a tabular report generated by the DSMON utility showing Network Interface information
- FIGURE 8 depicts a tabular report generated by the DSMON utility showing disk space information
- FIGURE 9 depicts a tabular report generated by the DSMON utility showing slot chain information
- FIGURE 10 depicts the DSFS file system organization
- FIGURE 11 depicts the use of standard forensic and analytical tools in conjunction with the INPCS appliance
- FIGURE 12 depicts the internal system architecture of the INPCS
- FIGURE 13 depicts the Disk Space Store Partition as a contiguous list of physical 64K clusters
- FIGURE 14 depicts the Disk Space Record in which logical slots are mapped on to physical devices
- FIGURE 15 depicts the slot cache buffers stored as contiguous runs
- FIGURE 16 depicts the use of a Name Table and Machine Table in a type 0x98 partition
- FIGURE 17 depicts the slot storage element layout comprising 64K clusters
- FIGURE 18 depicts the slot header and pointer system to the slot buffers containing data
- FIGURE 19 depicts sequential loading of slot cache elements on an LRU basis from an e1000 Adaptor Ring Buffer;
- FIGURE 20 depicts slot buffers allocated in a round-robin pattern from each buffer element in a slot buffer list
- FIGURE 21 depicts populated slot buffers in which the packets are of variable size and are efficiently stored so as to use all available buffer space in the slot cache element buffer chain;
- FIGURE 22 depicts the Slot Chain Table and Slot Space Table in schematic form
- FIGURE 23 depicts the internal layout depicted of the Slot Chain Table
- FIGURE 24 depicts the Space Table layout schematically
- FIGURE 25 depicts the storage of the Disk Space record and the Space Table linked to stored slots.
- FIGURE 26 depicts the on-disk slot cache segment chains employing a last recently uses LRU recycling method
- FIGURE 27 depicts the Allocation Bitmap and Chain Bitmap table structure
- FIGURE 28 depicts the use of a slot hash table to map slot LRU buffer elements
- FIGURE 29 depicts a request for reading or writing slot data from the volatile and non-volatile slot caches
- FIGURE 30 depicts Ethernet adaptors allocating slot LRU elements from cache
- FIGURE 31 depicts the recycling of the oldest entries as they are released;
- FIGURE 32 depicts the DSFS virtual file system;
- FIGURE 33 depicts the use of pjiandle context pointers in merging sots based on time domain indexing
- FIGURE 34 depicts the employment of pjiandle context structures via user space interfaces to create virtual network adapters that appear as physical adapters to user space applications;
- FIGURE 35 depicts the use of a filter table to include or exclude packet data from a slot cache element
- FIGURE 36 depicts a Virtual Interface mapped to a specific shot chain
- FIGURE 37 depicts the DSFS primary capture node mapped onto multiple archive storage partitions
- FIGURE 38 depicts the use of a mirrored I/O model to write data simultaneously to two devices using direct DMA
- FIGURE 39 depicts mirroring of captured data in a SAN (System Area Network) environment; and FIGURE 40 depicts the method for tagging captured packets.
- SAN System Area Network
- the INPCS is a high performance data capture recorder capable of capturing all network traffic present on a network or on multiple networks and archiving the captured data on a scalable, infinite, disk based LRU (least recently used) caching system, as is known in the art, at multiple gigabit (Gb) line speeds.
- INPCS has the ability to capture and stream to disk all network traffic on a gigabit Ethernet network and to present the data as a Virtual File System (VFS). End users may access information by retrieving it from the VFS to facilitate network security, forensics, compliance, analytics and network management applications as well as media applications utilizing video or audio formats.
- INPCS also supports this capability via T1/T3 and other topologies known in the art that utilize packet based encapsulation methods.
- the INPCS does not require the configuration of a protocol stack, such as TCP/IP, on the capture network device. This makes the INPCS "invisible” or passive and not addressable from the capture network segment. In this way, the device can't be targeted for attack since it can't be addressed on the network.
- the INPCS also provides a suite of tools to retrieve the captured data in time sequenced playback, as a virtual network interface or virtual Ethernet device, a regenerated packet stream to external network segments, or as a VFS that dynamically generates LIBPCAP (Packet Capture file format) and TCPDUMP (TCP protocol dump file format), CAP, CAZ, and industry standard formats that can be imported into any appropriate application that supports these formats.
- LIBPCAP is a system-independent interface for user-level packet capture that provides a portable framework for low-level network monitoring. Applications include network statistics collection, security monitoring, network debugging.
- the INPCS allows analysis of captured data while the device is actively capturing and archiving data.
- FIG. 1 depicts one embodiment of the hardware configuration of the integrated INPCS appliance .
- the INPCS platform is rack mountable device that supports large amounts of RAID 0/RAID 5/RAID 0+1 and RAID 1 disk storage with high performance Input/Output (I/O) system architectures.
- the INPCS device can sustain capture and storage rates of over 350 MB/s (megabytes per second).
- the device can be attached to Ethernet networks via, copper or SX fiber via either a SPAN port (port mirrored) 101 router configuration or via an optical splitter 102.
- SPAN port port
- optical splitter 102 optical splitter 102.
- multiple sources of network traffic including gigabit Ethernet switches 103 may provide parallelized data feeds to the capture appliance 104, effectively increasing collective data capture capacity.
- Multiple captured streams of data are merged into a consolidated time indexed capture stream to support asymmetrically routed network traffic as well as other merged streams for external consumption.
- the merged data stream is archived to an FC-AL
- the FC-AL switch 105 shown in Figure 1 offers eight ports with dedicated non-blocking 100 MB/second or 1 GB/second point-point parallel connections. These ports direct the captured network traffic to multiple FL-AL RAID Arrays 106.
- the depicted arrays each provide a total storage capacity of 7 Terabyte and may be configured using standard RAID configurations as known in the art.
- the present embodiment provides a controller that supports RAID 0 (striping without redundancy) or RAID 5 (distributed parity), RAID 0+1 (mirrors with stripes), RAID 1 (mirrors) as the preferred storage modes.
- FIG. 2 depicts a typical appliance chassis (2U configuration) designed to hold up to 8 standard 3-inch hard disk drives, and the associated hardware, firmware, and software.
- each chassis would contain eight 400GB hard disk drives for a total storage capacity of 3.2 Terabytes per chassis.
- the INPCS platform is a UL/TUV and EC certified platform and is rated as a Class A FCC device.
- the INPCS unit also meets TUV-1002, 1003,1004, and 1007 electrostatic discharge immunity requirements and EMI immunity specifications.
- the INPCS platform allows console administration via SSH (Secure Shell access) as well as by attached atty and tty serial console support through the primary serial port ensuring a secure connection to the device.
- SSH Secure Shell access
- the unit supports hot swapping of disk drives and dynamic fail over of IDE devices via RAID 5 fault tolerant configuration.
- the unit also supports a high performance RAID 0 array configuration for supporting dual 1000 Base T (1Gb) stream to disk capture.
- Captured network traffic stored on the SAN can be exposed to external appliances and devices or appropriate applications running on the INPCS appliance utilizing three primary methods: a VFS filesystem exposing PCAP formatted files, a virtual network interface (Ethernet) device and through a regenerated stream of packets to external network segments feeding external appliances.
- the INPCS file system acts as an on-disk LRU (least recently used) cache and recycles the oldest captured data when the store fills and allows continuous capture to occur with the oldest data either being recycled and overwritten or transferred to external storage for permanent archive of captured network traffic. This architecture allows for an infinite capture system.
- VFS files are dynamically generated by an implemented Linux VFS, known in the art, that resides on top of the disk LRU that INPCS employs to capture network traffic to the disk. Since INPCS presents data via a standard VFS, this allows this data to be easily imported or accessed by applications or to be exported to other computer systems on using network standards such as scp (secure copy), HTTPS (secure Hyper Text Transport Protocol), SMB (Microsoft's Server Message Block protocol) or NFS (the Unix Network File System protocol. This allows the INPCS device to be installed in a wide range of disparate networking environments. Additionally, exposing the captured network traffic through a filesystem facilitates transfer or backup to external devices including data tapes, compact discs (CD), and data DVDs. A filesystem interface for the captured traffic allows for easy integration into a wide range of existing applications that recognize and read such formats.
- the INPCS allows the archived data to be accessed as Virtual Network Interface using standard Ethernet protocols. Many security, forensics and network management applications have interfaces that allow them to open a network interface card directly, bypassing the operating system. This allows the application to read packets in their "raw" form from the network segment indicated by the opened device.
- the INPCS virtual internet device may be mapped onto the captured data store such that the stored data appear to the operating system as one or more physical network devices and the time-stamped stored data appears as if it were live network traffic. This allows existing applications to mimic their inherent direct access to network interface devices but with packets fed to the device from the captured packets in the INPCS. This architecture allows for ready integration with applications that are designed to access real-time network data, significantly enhancing their usability by turning them into tools that perform the same functions with historical data.
- the Virtual Network Interface also allows analysts to configure the behavior of the INPCS virtual Ethernet device to deliver only specific packets desired. For example, since the INPCS device is a virtual device a user may program its behavior. Tools are provided whereby only packets that meet predetermined requirements match a programmed filter specification (such as by protocol ID or time domain). Additionally, while physical Ethernet devices that are opened by an application are rendered unavailable to other applications, the virtual interface employed by INPCS allows for multiple applications to read from virtual devices (which may be programmed to select for the same or different packet subsets) without mutual exclusion and without any impact on real-time network performance.
- the INPCS also facilitates data access through regeneration. Captured packets in the INPCS store can be re-transmitted to external devices on attached network segments. This allows for a "regeneration" of packets contained in the store to be sent to external appliances, emulating the receipt of real-time data by such appliances or applications.
- the INPCS includes tools to program the behavior of regeneration. For instance, packets can be re-transmitted at defined packet rates or packets that meet particular predetermined criteria can be excluded or included in the regenerated stream.
- External appliances receiving packets regenerated to them by the INPCS appliance are unaware of the existence of the INPCS appliance, thus integration with existing or future appliances is seamless and easy, including applications where confidentiality and security are of paramount importance.
- This regeneration method also facilitates "load balancing" by retransmitting stored packet streams to external devices that may not be able to examine packets received into the INPCS appliance at the real-time capture rate. Additionally, this method can make external appliances more productive by only seeing packets that a user determines are of interest to current analysis. Regeneration has no impact on the primary functions of the INPCS as it can be accomplished while the INPCS appliance is continuing to capture and store packets from defined interfaces.
- the INPCS file system acts as an on-disk LRU (least recently used) cache, as is known in the art and recycles the oldest captured data when the store fills and allows continuous capture to occur with the oldest data either being recycled and overwritten or pushed out onto external storage for permanent archive of capture network traffic. This architecture allows for an infinite capture system. Captured packets at any given time in the on-disk store represents a view in time of all packets captured from the oldest packets to the newest.
- the INPCS software is implemented as loadable modules loaded into a modified Linux operating system kernel. This module provides and implements the VFS, virtual network device driver (Ethernet), and the services for regeneration of packets to external network segments, as described above. INPCS uses a proprietary file system and data storage. The Linux drivers utilized by the INPCS modules have also been modified to support a copyless DMA switch technology that eliminates all packet copies. Use of the copyless receive and send methodology is essential to achieving the desired throughput of the INPC. Copyless sends allow an application to populate a message buffer with data before sending, rather than having the send function copy the data.
- Captured packets are DMA (direct memory access) transferred directly from the network ring buffers into system storage cache without the need for copying or header dissection typical of traditional network protocol stacks. Similar methods are used for captured packets scheduled for writing to disk storage. These methods enable extremely high levels of performance and allows packet data to be captured and then written to disk at speeds of over 350 MB/s and allows support for lossless packet capture on gigabit networks. This enables the INPCS unit to capture full line rate gigabit traffic without any packet loss of live network data. This architecture allows real time post analysis of captured data by applications such as the popular Intrusion Detection System (IDS) software Snort, without the loss of critical data (packets). Additionally, should further research be desired, such as for session reconstruction, the full store of data is available to facilitate error free reconstruction.
- IDS Intrusion Detection System
- INPCS network troubleshooting and network forensics and analysis since it allows analysts an unparalleled view of live network traffic and flow dynamics. Since the unit captures all network traffic, it is possible to replay any event in time which occurred on a network.
- the device creates, in essence, a monolithic "network buffer" that contains the entire body of network traffic.
- INPCS exposes the capture data via a VFS file system (DSFS) as PCAP files.
- DSFS VFS file system
- the mounted DSFS file system behaves like traditional file systems, where files can be listed, viewed, copied and read. Since it is a file system, it can be exported via the Linux NFS or SMBFS to other attached network computers who can download the captured data as a collection time-indexed slot files or as consolidated capture files of the entire traffic on a network. This allows analysts the ability to simply copy those files of interest to local machines for local analysis.
- These capture PCAP files can also be written to more permanent storage, like a CD, or copied to another machine.
- the INPCS File System also creates and exposes both time- replay based and real-time virtual network interfaces that map onto the capture packet data, allowing these applications to process captured data in real time from the data storage as packets are written into the DSFS cache system.
- This allows security applications, for instance, to continuously monitor capture data in real time and provide IDS and alert capability from a INPCS device while it continues to capture new network traffic without interruption.
- This allows existing security, forensics, compliance, analytics and network management applications to run seamlessly on top of the INPCS device with no software changes required to these programs, while providing these applications with a lossless method of analyzing all traffic on a network.
- the INPCS unit can be deployed as a standalone appliance connected either via a Switched Port Analyzer (SPAN) or via an optical splitter via either standard LX or SX fiber optic connections.
- the unit also supports capture of UTP- based Ethernet at 10/100/1000 Mb line rates.
- the INPCS unit can also be configured to support asymmetrically routed networks via dual SX fiber to gigabit Ethernet adapters with an optical splitter connecting the TX/RX ports to both RX ports of the INPCS device.
- the INPCS unit is connected to a router, then the router is configured to mirror selected port traffic into the port connected to the INPCS Unit.
- Figure 3 depicts schematically the use of the INPCS appliance in a SPAN configuration.
- the INPCS appliance is connected to a router port, and the router is configured to mirror (i.e. to copy) packets from other selected ports to the SPAN configured port on the host router. This method does degrade performance of the router to some extent, but is the simplest and most cost effective method of connecting a INPCS appliance to a network for monitoring purposes.
- One distinct advantage of using a SPAN configuration relates to multi- router networks that host large numbers of routers in a campus-wide networked environment such as those that exist at universities or large business establishments.
- Routers can be configured to mirror local traffic onto a specific port and redirect this traffic to a central router bank to collect data on a campus- wide wide basis and direct it to a specific router that hosts an INPCS data recording appliance.
- This deployment demonstrates that even for a very large network utilizing gigabit Ethernet segments, this method is both deployable, and practical.
- average network traffic in and out of the university may be expected to continue at a sustained rate of approximately 55 MB/s with peaks up to 80 MB/s across multiple gigabit Ethernet segments.
- a deployment of the INCPS appliance utilizing a SPAN configuration can be effected without noticeable effect on the network and the INCPS can readily capture all network traffic at these rates and thus keep up with capture of all network traffic in and out of the university or similar sized enterprise.
- the INPCS appliance can be configured to support capture of network traffic via an in-line optical splitter that diverts RX (receive) and TX (transmit) traffic in a configuration that feeds into two SX gigabit Ethernet adapters within the INPCS appliance.
- Figure 4 depicts the use of the INPCS appliance in such an asymmetric routed configuration.
- the INPCS appliance is connected to an optical splitter that supports either SX (multi-mode) or LX (single mode long haul) fiber optic gigabit cables.
- This method provides very high levels of performance and is non-intrusive. The non-intrusive nature of this configuration method renders the INPCS appliance totally invisible on the customer network since the unit is completely shielded from view of any outside network devices.
- the INPCS appliance supports both of these modes and also provides the ability to present the view of the captured network traffic as a merged and consolidated chain of captured packets.
- Figure 5 shows the INPCS appliance in an optical splitter configuration.
- the INPCS supports only SX fiber in the appliance chassis.
- optical splitters and converters may be added to the configuration to allow LX to SX fiber connections via an external network tap device.
- the INPCS provides several utilities that allow configuration of virtual interfaces, starting and stopping data capture on physical adapters, mapping of virtual network interfaces onto captured data in the data store, and monitoring of network interfaces and capture data status.
- the entire captured data store is exported via a virtual file system that dynamically generates LIBPCAP files from the captured data as it is captured and allows these file data sets to be viewed and archived for viewing and forensic purposes by any network forensics programs that support the TCPDUMP LIBPCAP file formats for captured network traffic.
- the DSCAPTURE utility configures and initiates capture of network data and also allows mapping of virtual network interfaces and selection of specific time domains based on packet index, date and time, or offset within a captured chain of packets from a particular network adapter or network segment.
- the utility provides the following functions as they would appear in a command line environment:
- the function DSCAPTURE INIT will initialize the INPCS capture store.
- DSCAPTUE START and DSCAPTURE STOP start and stop packet capture of network traffic, respectively, onto the local store based on network interface name.
- Linux names interfaces ethO, eth1 , eth2, etc. such that control code would resemble the following:
- the DSCAPTURE MAP and DSCAPTURE MAP SHOW functions allow specific virtual network interfaces to be mapped from physical network adapters onto captured data located in the store. This allows SNORT, TCPDUMP, ARGUS, and other forensic applications known in the art to run on top of the INPCS store in a manner identical to their functionality were running on a live network adapter. This facilitates the use of a large number of existing or custom- designed forensic applications to concurrently analyze captured traffic at near real-time performance levels.
- the virtual interfaces to the captured data emulating a live network stream will generate a "blocking" event when they encounter the end of a stream of captured data from a physical network adapter and wait until new data arrives. For this reason, these applications can be used in unmodified form on top of the INPCS store while traffic is continuously captured and streamed to these programs in real time with concurrent capture of network traffic to the data store, as shown in the following command line sequence:
- the DSCAPTURE function also allows the mapping of specific virtual interfaces to physical interfaces as shown in the following command line sequence and display:
- the ifp ⁇ #> and ift ⁇ #> named virtual network interfaces
- the ifp ⁇ #> named virtual interfaces provide the ability to read data from the data store at full rate until the end of the store is reached.
- the ift ⁇ #> named virtual interfaces provide time sequenced playback of captured data at the identical time windows the data was captured from the network.
- This second class of virtual network interface allows data to be replayed with the same timing and behavior exhibited when the data was captured live from a network source. This is useful for viewing and analyzing network attacks and access attempts as the original timing behavior is fully preserved.
- the DSCAPTURE function also allows the virtual network interfaces to be indexed into the store at any point in time, packet number, or data offset a network investigator may choose to review, as in the follow command line sequence:
- the virtual network interface pointer into the capture stream at a specific location.
- the virtual device When the virtual device is then opened, it will begin reading packets from these locations rather that from the beginning of the capture stream.
- the DSMON utility allows monitoring of a INPCS device from a standard Linux console, atty, or xterm window connected to the device via serial port, SSH (Secure Shell Login) , or via a Terminal Window via an xterm device as is known in the art.
- This program provides comprehensive monitoring of data capture status, captured data in the store, network interface statistics, and virtual interface mappings.
- Figure 6 depicts menu options for DSMON function screen console. The user may select and view information pertaining to network interfaces, slot cache, disk storage, slot chains, available virtual interfaces, and merged chains.
- the DSMON utility supports monitoring of all network interfaces and associated hardware statistics, including dropped packet, FIFO and frame errors, receive packet and byte counts, etc.
- This utility also monitors cache usage within the system, disk storage usage, a capture monitor that records malformed packets, total captured packets, disk channel I/O performance statistics, slot chain information including the mapping of slot chains to physical network interfaces, the number of slots chained to a particular adapter, the dates and time packet chains are stored in slots and their associated chains, virtual interface mappings, virtual interface settings, and merged slot chains for support of asymmetric routed captured traffic, traffic captured and merged from optical splitter configurations.
- Figure 7 depicts a typical tabular report generated by the DSMON utility showing the status of the Network Interface.
- the display provides comprehensive information regarding the identify of the Network Interface, the device type, internet address, hardware address, broadcast type, maximum transmission unit (MTU) setting, interrupt status, line/link status, packet receive rate, byte receive rate, maximum burst rate for packets and bytes received, packets dropped, total packets and bytes captured, and dropped buffers.
- MTU maximum transmission unit
- Figure 9 depicts a typical tabular report generated by the DSMON utility showing the status of the slot chain, each slot representing a pre-determined segment of captured data.
- the display provides information regarding the INPCS up time, active slot chains and their start times and sizes.
- the INPCS data recorder exposes captured data via a custom Virtual File
- DSFS Dynamically generates LIBPCAP formatted files from the slots and slot chains in the data store.
- This data can be accessed via any of the standard file system access methods allowing captured data to be copied, archived and reviewed or imported into any programs or applications that support the LIBPCAP formats.
- the INPCS system exposes a new file system type under the Linux Virtual File System (VFS) interface as follows:
- the DSFS registers as a device based file system and is mounted as a standard file system via the mount command under standard System V Unix systems and systems that emulate the System V Unix command structure.
- This file system can be exposed to remote users via such protocols as NFS, SAMBA, InterMezzo, and other remote file system access methods provided by standard distributions of the Linux operating system. This allows the DSFS file system to be remotely access from Windows and Unix workstation clients from a central location.
- DSFS appears to the operating system and remote users as simply another type of file system supported under the Linux Operating System, as shown in the command line sequence below:
- Figure 10 depicts the DFS file system structure schematically.
- the DSFS file system is a read only file system from user space. However, it does support chmod and chown commands to assign specific file permissions to designated end users of the system. This allows a central administrator to allow selected individuals to access files contained in the DSFS file system on an individual basis, allowing greater freedom to configure and administer the system if it is intended to be used by a Network Security Office that has more than one Network Forensic Investigator. Only the underlying capture engine subsystem can write and alter data in the DSFS file system. Beyond the assignment of user permissions to specific files, DSFS prohibits alteration of the captured data by any user, including the system administrator. This ensures the integrity of the captured data for purposes of chain of custody should the captured data be used in criminal or civil legal proceedings where rules of evidence are mandatory.
- the DSFS File System is organized into the following directory structure:
- DSFS exposes captured slot chains in the root DSFS directory by adapter number and name in the system as a complete chain of packets that are contained in a LIBPCAP file. If the captured adapter contains multiple slots within a chain, the data is presented as a large contiguous file in PCAP format with the individual slots transparently chained together. These files can be opened either locally or remotely and read into any program that is designed to read LIBPCAP formatted data.
- These master slot chains are in fact comprised of sub chains of individual slots that are annotated by starting and ending date and time.
- the master slot chain files can also be imported from the root DSFS directory in the same manner and can be copied and archived as simple system files to local or remote target directories for later forensic analysis, as shown in the following command line example:
- the DSFS "stats" directory contains text files that are dynamically updated with specific statistics information similar to the information reported through the DSMON utility. These files can also be opened and copied; thereby, providing a snapshot of the capture state of the INPCS system for a particular time interval, as shown:
- the file slot.txt contains the current cache state of all slot buffers in the DSFS system and can be displayed and copied as a simple text file with the following command line sequence:
- Network Interface Io (1) active slot 0/00000000 packets-0 ringbufs-0 total_bytes-0 metadata-0
- Network Interface sitO (2) active slot 0/00000000 packets-0 ringbufs-0 total_bytes-0 metadata-0 Network Interface : eth ⁇ (11) active slot 0/00000000 packets-0 ringbufs-0 total_bytes-ONetwork Interface : eth1 (12) active slot 1/728A0000 packets-1177 ringbufs-512 total_bytes-125125 metadata-65912
- HASHED slot 0000000/7279C000 i:12 l:00
- VALID UPTD HASHED slot 0000000/72798000 i:00 l:00
- FREE slot 0000000/72794000 i:00 l:00
- FREE slot 0000000/72790000 i:00 1:00
- FREE slot 0000000/7278C000 i:00 1:00
- FREE slot 0000000/72788000 i:00 1:00
- FREE slot 0000000/72784000 i:00 1:00
- FREE slot 0000000/72780000 i:00 1:00
- FREE slot 0000000/7277C000 i:00 1:00
- FREE slot 0000000/72778000 i:00 1:00
- FREE slot 0000000/72774000 i:00 1:00
- FREE slot 0000000/72770000 i:00 1:00
- FREE slot 0000000/7276C000 i:00 1:00
- FIG. 11 depicts the use of the INPPCS in conjunction with a number of standard network analysis and forensic tools known in the art.
- TCPDUMP can be configured to run on top of INPCS by utilizing Virtual Network Interfaces, as in the following command line sequence:
- the SNORT Intrusion Detection System can be run with no software changes on top of the INPCS data recorder through the same use of the virtual network interfaces provided by the INPCS appliance. Since the Virtual Interfaces block when they reach the end of store data, SNORT can run in the background in real time reading from data captured and stored in a INPCS appliance as it accumulates. The procedure for invoking and initializing SNORT appears as shown in the following command line sequence and display:
- Fragment timeout 60 seconds
- Fragment memory cap 4194304 bytes
- Fragment ttljimit 5 Fragment Problems: 0
- Session memory cap 8388608 bytes State alerts: INACTIVE
- Async Link 0 State Protection: 0 Self preservation threshold: 50 Self preservation period: 90 Suspend threshold: 200 Suspend period: 30
- Stream4_reassemble config Server reassembly: INACTIVE Client reassembly: ACTIVE Reassembler alerts: ACTIVE Zero out flushed packets: INACTIVE flush_data_diff_size: 500 Ports: 21 23 25 53 80 110 111 143 513 1433 Emergency Ports: 21 23 25 53 80 110 111 143 513 1433 Httplnspect Config: GLOBAL CONFIG
- YES alert NO IIS Backslash: YES alert: NO Directory: YES alert: NO Apache WhiteSpace: YES alert: YES IIS Delimiter: YES alert: YES
- I memory-cap 1048576 bytes
- Figure 12 depicts the internal system architecture of the INPCS.
- the invention is designed as a high speed on-disk LRU cache of storage segments that are treated as non-volatile (written to disk) cache segments that capture and store network traffic at gigabit per second line rates.
- the architecture is further enhanced to provide the ability to stripe and distribute slot cache segments across multiple nodes in a storage cluster utilizing Fiber Channel or 10GbE (10 gigabit)(iSCSI) Ethernet networking technology.
- Slot Storage segments are allocated and maintained in system memory as large discrete cache elements that correspondingly map to a cluster based mapping layer in system storage. These slot cache segments are linked into long chains or linked lists on non-volatile (disk) storage based upon the network interface for which they contain packets and network payload data captured from a particular network segment.
- the invention also allows rapid traffic regeneration of the captured data and retrieval of captured data via standard file system and network device interfaces into the operating system.
- This flexible design allows user space applications to access captured data in native file formats and native device support formats without the need for specialized interfaces and APIs (application programming interfaces).
- Each slot cache segment is time based and has a start time, end time, size, and chain linkage meta tag and are self annotated and self describing units of storage of network traffic. As the slot cache storage system fills with fully populated slot cache segments, older segments in a slot chain are overwritten or pushed/pulled into long term archive storage.
- the invention uses two primary disk partition types for the storage and archival of captured network traffic.
- partition type 0x97, 0x98 and partition type 0x99 are known in the art.
- Partition type 0x97 partitions are used by the system to storage active data being captured from a live network medium.
- Partition type 0x98 partitions are long term storage used to archive captured network traffic into large on-disk library caches that can span up to 128 Tera-bytes of disk storage for each Primary capture partition.
- Type 0x97 partitions are described by a Disk Space Record header located on each partition.
- the Disk Space Record Header describes the block size, partition table layout, and slot storage layout of a type 0x97 partition.
- the Disk Space Record Header uses the following on-disk structure to define the storage extents of either a type 0x97 or type 0x98 storage partition.
- Disk Space Records also allow chaining of Disk Space Records from multiple type 0x97 or type 0x98 partitions based upon creation and membership ID information stored in a membership cluster map, which allows the creation of a single logical view of multiple type 0x97 partitions. This allows the system to concatenate configured type 0x97 partitions into stripe sets and supports data striping across multiple devices, which increases disk channel performance dramatically.
- Disk Space Records also define the internal table layouts for meta-data and chaining tables used to manage slot cache buffer chains within a virtual Disk Space Record set.
- Disk Space records contain table pointers that define the tables used by the DSFS file system to present slot storage as logical files and file chains of slot storage elements.
- Disk Space Record based storage divides the storage partition into contiguous regions of disk sectors called slots. Slots can contain from 16 up to 2048 64K blocks of 512 byte sectors, and these storage elements are stored to disk in sequential fashion. Slots are access via a sequential location dependent numbering scheme starting at index 0 up to the number of slots that are backed up by physical storage on a particular disk device partition.
- Each Disk Space Record contains a space table.
- the space table is a linear listing of structures that is always NUMBER_OF_SLOTS * sizeof (S PAC E_TABLE_E NTRY) in size.
- the Space table maintains size, linkage, and file attribute information for a particular slot and also stores the logical chaining and ownership of particular slots within a logical slot chain.
- Figure 13 depicts the Disk Space Store Partition that is addressed as a contiguous list of physical 64K clusters.
- a cluster is defined as a 64K unit of storage that consists of 128 contiguous 512 byte sectors on a disk device.
- DSFS views partitions as linear lists of cluster based storage, and storage addressing is performed on the unit of a cluster for partition type 0x97 and 0x98. All disk addresses are generated and mapped based on a logical 64K cluster unit of storage and caching. Slots are comprised of chains of 64K buffers that correspondingly map to 64 cluster addresses on a Disk Space Store partition or a Virtual Disk Store Partition.
- Disk Space Records that perform striping use an algorithm that round robins the cluster address allocation between the various partitions that comprise a DSFS Disk Space Record member stripe set.
- the module of a cluster number relative to the number of stripe members is performed and used as an index into a particular disk LBA offset table of partition offsets within a disk device partition table that calculates the relative LBA offset of the 64K cluster number.
- Cluster numbers are divided by the number of striped members to determine and physical cluster address and sector LBA offset into a particular stripe set partition.
- Figure 14 depicts the Disk Space record in which logical slots are mapped on to physical devices.
- the Disk Space record is always the first storage sector in a DSFS partition.
- Storage sectors in a DSFS partition are always calculated to align on configured I/O block size (4K) page boundaries.
- I/O block size (4K) page boundaries There are instances where a partition can be created that does not align on a 4K boundary relative to LBA sector addressing.
- DSFS partitions are always adjusted to conform with aligned block addressing relative to LBA 0 if a partition has been created that is not block aligned.
- the algorithm performing this addressing alignment uses the following calculation to enforce I/O block size (4K) alignment:
- the Disk Space Record will occupy the first cluster of an adjusted Disk Space Record partition.
- the DSR records the cluster offset into the virtual Disk Space Store of the location of the Space Table, and optionally for partition type 0x98, the Name and Machine Tables as well.
- the DSR also contains a table of slot chain head and tail pointers. This table is used to create slot chains that map to physical network adapters that are streaming data to the individual slot chains. This table supports a maximum of 32 slot chains per Disk Space Record Store. This means that a primary capture partition type 0x97 can archive up to 32 network adapter streams concurrently per active Capture Partition.
- Type 0x98 Archive Storage Partitions employ a Name Table and Machine table that are used to store slots from primary capture partitions for long term storage and archive of network traffic and also record the host machine name and the naming and meta-tagging information from the primary capture partition. depicts the use of a Name Table and Machine Table in a type 0x98 partition.
- the interface name and machine host name are added to the name table and the host name table on the archive storage partition.
- This allow multiple primary capture partitions to utilize a pool of archive storage to archive captured network traffic from specific segments into a large storage pool for archival and post capture analysis.
- Archive storage can be mapped to multiple Network Capture Appliances as a common pool of slot segments.
- Archive storage pools can also be subdivided into storage zones with this architecture and tiered as a hierarchical cache and archive network traffic for months, or even years from target segments.
- Disk Space Store Individual Slot addresses are mapped to the Disk Space Store based upon partition size, number of slots, storage record cluster size, and reserved space based on the following algorithm:
- slot_cluster (disk space record->start_of_slot_data +
- the Start of slot data is the logical cluster address that immediately follows the last cluster of the space table for type 0x97 partitions and the last cluster of the machine table for type 0x98 partitions. Slots are read and written as a contiguous run of sectors to and from the disk storage device starting with the mapped slot cluster address derived from the slot number.
- a slot defines a unit of network storage and each slot contains a slot header and a chain of 64K clusters.
- the on-disk structure of a slot is identical to the cache in-memory structure and both memory and the on-disk slot caches are viewed and treated by DSFS as specialized forms of LRU (last recently used) cache.
- the slot header stores meta-data that describes the content and structure of a slot and its corresponding chain of 64 clusters.
- Figure 17 depicts the slot storage element layout comprising 64K clusters.
- the slot header points to the buffers as a character byte stream and also maintains starting index:offset pairs into buffer indexes within a slot.
- Figure 18 depicts the slot header and pointer system to the slot buffers containing data. Buffers in a slot are indexed zero relative to the first buffer element contained in a slot buffer segment.
- a slot can have from 16-2048 buffer elements. Slots also provide a block oriented method for packet traversal that allow network packets to be skipped over based on index:offset pair. This index:offset pair is handled by the file system layers as a virtual index per packet into a slot segment.
- the slot buffer header points to the first index:offset and the last index:offset pair within a slot segment buffer, and also contains a bitmap of buffer indexes that are known to contain valid slot data. These indexes are used by the I/O caching layer for reading sparse slots (slots not fully populated with network packet data) into memory efficiently.
- the high performance of this invention is derived from the technique described for filling of pre-load addresses into a network adapter device ring buffer.
- Network adapters operate by pre-loading an active ring or table on the adapter with memory addresses of buffer addresses to receive incoming network packets. Since the adapter cannot know in advance how large a received packet may be, the pre-loaded addresses must be assumed to be at least as large as the largest packet size the adapter will support.
- the algorithm used by DSFS always assumes at least the free space of (PACKET_SIZE +1) must be available for a pre-load buffer since buffers can exceed the maximum packet size due to VLAN (Virtual LAN) headers generated by a network router or switch.
- VLAN Virtual LAN
- the network adapter allocates buffers from the DSFS slot cache into the adapter based upon the next available index:offset pair.
- the buffers are maintained as a linear list of index addresses that are cycled through during allocation that allows all ring buffer entries to be pre-loaded from a buffer array (i.e. slot segment) in memory. The number of slot buffers must therefore be
- the buffer header is pinned in memory for that particular buffer, and subsequent allocation requests will skip this buffer until the pre-loaded element has been received from the adapter.
- Slot buffers are allocated in a round-robin pattern from each buffer element in a slot buffer list, as depicted in Figure 20.
- Linkages are maintained between each element into the next buffer that are accessed by means of an index:offset pair as described. These comprise a coordinate address for a buffer location of stored data and allow the lost buffer to preload capture addresses into the ring buffers of a capture device that supports direct DMA access at very high data rates into a slot buffer element cached in memory. Reading the captured data requires that the slot be held in memory and the elements traversed via a set of linkages within each element header that point to the next index:offset address pair for a stored element or network packet.
- the allocation algorithm is as follows:
- L_DIRTY;
- slot->current_buffer % slot->d->buffer_count
- slot->b->starting_offset buffer->header_offset
- element->previous_offset O
- element->previous_index OxFFFFFFFF
- element->next_offset O
- element->next_index OxFFFFFFFF
- ⁇ buffer (slot->buffers[slot->current_buffer % slot->d- >buffer_count]); if (Ibuffer)
- Figure 21 depicts an example of populated slot buffers in which the packets are of variable size and are efficiently stored so as to use all available buffer space in the slot cache element buffer chain. This is achieved assigning bugger allocations from allocated preload buffers until the adaptor releases that buffer through a receive interrupt and posts the size of the received packet. The buffer is then set to the next index:offset pair and flagged as available for preload allocation into the adapter ring buffer.
- This approach allows network packets to be tightly packed using the full amount of available slot cache buffer memory with little waste. This improves capture line rates by using disk storage space and reducing the write size overhead for captured data. With this model, data captured from the network in terms of bytes/second is more accurately reflected as the actual writes sizes of data written through the disk I/O channel.
- the Disk Space Record contains a 32 entry slot chain table.
- the Slot chain table defines the starting and ending slot Identifiers for a chain of populated slot cache elements that reside in the non-volatile system cache (on-disk).
- the Slot Chain table also records the date extents for capture network packets that reside in the time domain that comprises the sum total of elapsed time between the starting and ending slot chain element. As slots are filled, each slot records the starting and ending time for the first and last packet contained within the slot cache element. Slots internally record time at the microsecond interval as well as UTC time for each received packet, however, within the Slot Chain and Space Table, only the UTC time is exported and recorded since microsecond time measurement granularity is not required at these levels for virtual file system interaction.
- the Slot Chain Table uses the internal layout depicted in Figure 23 to record specific information about each allocated slot chain.
- the disk space record contains a slot chain table the records the starting and ending slot index for a slot chain of captured elements. This table also records the number of slots in a chain and the starting and ending date:time for data stored in a linked chain of slots.
- the Slot Chain Table records the starting slot address for a slot chain, the ending slot address for a slot chain, the number of total slots that comprise a slot chain, and the starting and ending dates for a slot chain. The dates are stored in standard UTC time format in both the Slot Chain Table and the System Space Table.
- the slot chain table is contained within these fields in the disk space record header: ULONG slot_starting_cluster[MAXJNTERFACE_SLOTS]; ULONG slot_ending_cluster[MAXJNTERFACE_SLOTS]; ULONG slot_starting_time_domain[MAXJNTERFACE_SLOTS]; ULONG slot_ending_time_domain[MAXJNTERFACE_SLOTS];
- the Space Table serves as the file allocation table for Slot Chains in the system.
- Figure 24 depicts the Space Table layout schematically. Slot Chains are analogous to files in a traditional file system.
- the Space table contains a field that points to the next logical slot within a slot chain, as well as starting and ending dates in UTC time format for packets stored within a Slot Cache Element.
- the space table also stores meta-data used for dynamic file reconstruction that includes the number of packets stored in a slot cache element, the number of total packet bytes in a slot cache element, file attributes, owner attributes, meta-data header size, and the size of packet sliced bytes (96 byte default).
- WORD interface WORD interface; umode_t mode; uid_t u id; gidj gid; long long size;
- Space Table Linkages are created by altering the next slot field which corresponds to a slot on a Disk Space Record Store.
- the Space Table entries are sequentially ordered based on slot position within the store. Index 0 into the Space Table corresponds to slot 0, index 1 to slot 1 , and so forth.
- Space Table information is mirrored in both a secondary Mirrored Space table, and also exists within the slot cache element header for a slot as well. This allows a Space Table to be rebuilt from slot storage even if both primary and secondary Space Table mirrors are lost and is provided for added fault tolerance.
- the slot number address space is a 32-bit value for which a unique disk space record store is expressed as:
- OxFFFFFFFF is reserved as an EOF (end of file) marker for the Space Table next slot entry field which allows a range of 0 - (OxFFFFFFFF -1) permissible slot addresses.
- Slot Chains are created and maintained as a linked list in the Space Table of slots that belong to a particular slot chain. The beginning and ending slots and their time domain and ending domain values are stored in the Slot Chain table in the DSR, and the actual linkages between slots is maintained in the space table. During Space Table traversal, when the value OXFFFFFFFFFF is encountered, this signals end of chain has been reached.
- the DSFS space table maintains an allocation table that employs positional chain elements in a forward linked list that describe a slot index within a DSFS file system partition.
- the Disk Space record stores the actual cluster based offset into a DSFS partition for meta-table and slot storage.
- Figure 25 depicts the storage of the Disk Space record and the Space Table linked to stored slots. This example illustrates a slot chain comprising elements 0-4. Space Table index 0 has a next slot entry of 1 , 1 points to 2, 2 to 3, 3 to 4, and 4 to OXFFFFFFFF.
- slots are allocated based upon a bit table built during DSR mount that indicated the next free slot available on a particular DSR. As slots are allocated, and the disk space record store becomes full, it becomes necessary to recycle the oldest slot cache elements from the store. Since the time domain information for a particular slot chain is stored in the Disk Space Record header, it is a simple matter to scan the 32 entries in the table and determine the oldest slot cache element reference in a slot chain head. When the slot cache has become completely full, the oldest slot segment is pruned from the head of the target slot chain and re-allocated for storage from the volatile (in-memory) slot element cache.
- FIG. 26 depicts the on-disk slot cache segment chains employing a last recently uses LRU recycling method.
- the starting slot located in the slot chain table is pruned from the slot chain head based on the oldest starting slot in the Slot Chain Table for a given Disk Space Record of slot cache storage segments.
- a DSFS disk space record store During initial mounting and loading of a DSFS disk space record store, the store is scanned, space tables are scanned for inconsistencies, and the chain lengths and consistencies are checked. During this scan phase, the system builds several bit tables that are used to manage allocation of slot cache element storage and chain management. These tables allow rapid searching and state determinations of allocations and chain location and are used by the DSFS virtual file system to dynamically generate file meta-data and LIBPCAP headers. These tables also enable the system to correct data inconsistencies and rapid- restart of due to incomplete shutdown.
- FIG. 27 depicts the Allocation Bitmap and Chain Bitmap table structure. After this table is constructed, DSFS verifies all the slot chain links and compares the allocations against a chain bitmap table that is annotated as each chain element is traversed. If a chain is found to have already been entered into the bitmap table, then a circular chain has been detected and the chain is truncated to a value of OxFFFFFFFF.
- Each Slot Chain Head maintains a bitmap of current slot allocations within it's particular chain. This table is used to validate slot membership within a chain by user space processes running about DSFS that may have stale handles or context into a chain after a recycle event.
- the Slot Chain bitmaps allow the DSFS virtual file system to verify a slots membership in a chain before retrying the read with a known slot offset location.
- the volatile (in-memory) slot element cache is designed as a memory based linked listing of slot cache elements that mirrors the slot cache element structure used on disk.
- the format is identical on-disk to the in-memory format that described a slot cache element. This list is maintained through three sets of linkages that are combined within the slot buffer header for a slot cache element.
- the structure of a slot cache element is as follows:
- the slot buffer header that describes a slot cache element is a member of four distinct lists.
- the first list is the master allocation list. This list maintains a linkage of all slot buffer heads in the system. It is used to traverse the slot LRU listing for aging of slot requests and write I/O submission of posted slots.
- the slot buffer header also can exist in a slot hash listing.
- Figure 28 depicts the use of a slot hash table to map slot LRU buffer elements. This listing is an indexed table that utilizes an extensible hashing algorithm to keep a hash of slots currently cached in the system. This allows rapid lookup of a slot by number from the system and is the main view portal from user space into the DSFS file system. If a slot does not exist in the hash listing with a valid ID, then it is not accessible during initial open operations of a slot.
- the LRU list is used by DSFS to determine which slot buffer header was touched last. More recent accesses to a slot buffer header result in the slot buffer header being moved to the top of the listing. Slot cache elements that have valid data and have been flushed to disk and have not been accessed tend to move to the bottom of this list over time. When the system needs to reallocate a slot cache element and it's associated slot buffer header for a new slot for either a read or write request to the volatile slot LRU cache, then the caching algorithm will select the oldest slot in memory that is not locked, has not been accessed, and has been flushed to disk and return date from it.
- Figure 29 depicts a request for reading or writing slot data from the volatile and non-volatile slot caches.
- a p_handle is used to submit a request to open a slot for reading network packets into user space applications. If the slot is already in memory, the p-handle opens the lost and reads packets until it reaches the end of slot data. If the slot is not in the LUR cache, the last recently used slot cache buffer is recycled and submits an asynchronous read to the disk to fill the slot from non-volatile (on-disk) cache storage.
- Network adapters that are open and capturing network packets allocate an empty slot buffer header which reference a slot cache element and its associated buffer chain from the LRU cache based on the algorithm depicted in Figure 30 which shows how adaptors allocate slot LRU elements from cache. These slot buffer headers are locked and pinned in memory until the adapter releases the allocated buffers. The system keeps track of allocated slot buffer headers through an adapter slot table that records the current active slot cache element that is being accessed by a particular adapter ring buffer.
- the slot LRU allows the network adapter at this layer to reallocate the same slot address in a unique slot buffer header and slot cache element. This process requires that the slot id be duplicated in the slot LRU until the last user space reference to a particular slot address is released. This even can occur if user space applications are reading data from a slot chain, and the application reaches a slot in the chain that has been recycled due to the slot store becoming completely full. In most cases, since slot chains contain the most recent data at the end of a slot chain, and the oldest data is located at the beginning of a slot chain, this is assumed to be an infrequent event.
- the newly allocated slot chain element in this case becomes the primary entry in the slot hash list in the LRU, and all subsequent open requests are redirected to this entry.
- the previous slot LRU entry for this slot address is flagged with a -1 value and removed from the slot hash list that removes it from the user space portal view into the DSFS volatile slot cache.
- the previous slot buffer header is evicted from the slot LRU and placed on a free list for reallocation by network adapters for writing or user space readers for slot reading by upper layer applications.
- Figure 31 depicts the recycling of the oldest entries as they are released.
- a single process daemon is employed by the operating system that is signaled via a semaphore when a slot LRU slot buffer header is dirty and requires the data content to be flushed to the disk array.
- This daemon uses the master slot list to peruse the slot buffer header chain to update aging timestamps in the LRU slot buffer headers, and to submit writes for posted LRU elements.
- an LRU slot buffer header can have the following states:
- Entries flagged as L_POST or L_REPAIR are written to non-volatile storage immediately. Entries flagged L_DIRTY are flushed at 30 second intervals to the system store. Meta-data updates to the Space Table for LJDIRTY slot buffer headers are synchronized with the flushing of a particular slot address. Slot buffer headers flagged L-LOADING are read requests utilizing asynchronous read I/O. L_HASHED means the slot address and slot buffer header are mapped in the slot hash list and are accessible by user space applications for open, read, and close requests.
- Figure 32 depicts the DSFS virtual file system. The DSFS Virtual File
- DSFS maps slots cache element as files and chains of slot cache elements as files to the user space operating system environment.
- DSFS also has the capability to expose this data in raw slot format, or dynamically generate LIBPCAP file formats to user space applications that use the file system interfaces.
- DSFS also exposes file system and capture core statistics as virtual files that can be read in binary and text based formats for external applications.
- the Virtual file system utilizes a virtual directory structure that allows a particular slot to expose multiple views of the slot data to user space.
- the directory layouts are all accessible via open(), read(), write(), lseek(), and closeO system calls; Slot chains are also exposed as virtual files and can also use standard system calls to read an entire slot chain of capture network traffic. LIBPCAP allows this data to be exported dynamically to a wide variety of user space applications and network forensics monitoring and troubleshooting tools.
- the DSFS file system utilizes a P_HANDLE structure to create a unique view into a slot cache element or a chain of slot cache elements.
- P-HANDLE structure records the network interface chain index into the Slot Chain table, and specific context referencing current slot address, slot index address, and offset within a slot chain, if a slot chain is being access and not an individual slot cache element.
- the PJHANDLE structure is described as:
- the PJHANDLE structure is also hierarchical, and allows PJHANDLE contexts to be dynamically mapped to multiple slot cache elements in parallel, that facilitates time domain based merging of captured network traffic.
- network TX/RX traffic may potentially be stored from two separate network devices that actually represent a single stream of network traffic.
- Figure 33 depicts the use of pjiandle context pointers in merging sots based on time domain indexing.
- the DSFS file system provide a specialized directory called the merge directory that allows user space application to create files that map PJHANDLE context pointers into unique views into a single capture slot chain, or by allowing user space applications to created a merged view of several slot chains that are combined to appear logically as a single slot chain.
- Commands are embedded directly into the created file name and parsed by the DSFS virtual file system and used to allocate and map PJHANDLE contexts into specific index locations within the specified slot chains.
- the format of the command language is more fully defined as:
- An interface number can also be used as an interface name. This was supported to allow renaming of interfaces while preserving the ability to read data captured on a primary partition including, by way of example, the following data sets and their respective command line entries:
- PJHANDLE context structures are also employed via user space interfaces to create virtual network adapters to user space that appear as physical adapters to user space applications as depicted in Figure 34.
- DSFS allows p_handle contexts to be mapped to the capture slot chain for a physical network adapter, such as ethO, and allow user space applications to read from the capture store as though it were a physical network.
- the advantage of this approach relates to packet lossless performance.
- the I/O subsystem in the DSFS capture system has been architected to favor network capture over user applications. Exporting virtual network interfaces allows user space intrusion detection systems to run as applications without being directly mapped to hardware devices.
- This also allows the user applications to process the captured network packets in the background while the network packets are streamed to the disk arrays in parallel. This provides significantly improved performance of intrusion detection applications without packet loss, since the application can simply sleep when the network load on the system becomes more active.
- p_handle KMALLOC(sizeof(P_HANDLE), GFP_KERNEL); if (Ipjiandle) return 0; memset(p_handle, 0, sizeof (P-HANDLE));
- new_p_handle KMALLOC(sizeof(P_HANDLE), GFP_KERNEL); if (!new_p_handle) break;
- new_p_handle->next NULL
- new_p_handle->prior NULL
- IOCTL calls to the virtual device return the next packet in the stream.
- the IOCTL call returns the oldest packet for the entire array of open slot chains.
- PJHANDLE contexts are unique and by default, are indexed to the current time the virtual interface is opened relative to the time domain position in a captured slot chain. This mirrors the actual behavior of a physical network adapter. It is also possible through the PJHANDLE context to request a starting point in the slot chain at a time index that is earlier or later than the current time a virtual interface was opened. This allows user space application to move backwards or forward in time on a captured slot chain and replay network traffic. Virtual interfaces can also be configured to replay data to user space applications with the exact UTC/microsecond timings the network data was actually received from the network segments and archived.
- Playback is performed in a slot receive event that is also hooked to the underlying operating system sys_recvmsg sockets call, calls to recvmsg redirect socket reads to the DSFS slot cache store and read from the mapped slot chain for a particular virtual interface adapter.
- the sys_recvmsg algorithm for redirecting operating system user space requests to read a socket from a virtual interface is described as:
- ⁇ new_p_handle get_merge_target(p_handle, NULL, NULL); if (!new_p_handle) return -ENOENT;
- ⁇ j_usec (ULONG)usec; schedule_timeout(j_usec / usec_per Jiffies);
- Virtual network interface mappings also employ an include/exclude mask of port/protocol filters that is configured via a separate IOCTL call and maps a bit table of include/exclude ports to a particular virtual network interface.
- Figure 35 depicts the use of a filter table to include or exclude packet data from a slot cache element. The algorithm that supports this will filter those network packets that do not match the search criteria from the sys_recvmsg socket based packet stream that is returned to user space applications. This allows virtual interfaces to be configured to return only packets that meet pre-determined port criteria, which is useful for those applications that may only need to analyze HTTP (web traffic).
- register int ip_hdr_len, s, d; unsigned char *data; struct iphdr *ip; struct tcphdr *tcp; struct udphdr *udp; register int ie_ret 1;
- case ETH_P_802_3 case ETH_P_802_2: return ie ⁇ ret;
- ⁇ d ntohs(tcp->dest); if (bitmap->bitmap[d » 3] & (1 « (d & 7))) ⁇
- ⁇ d ntohs(tcp->dest); if (bitmap->bitmap[d » 3] & (1 « (d & 7)))
- ⁇ d ntohs(udp->dest); if (bitmap->bitmap[d » 3] & (1 « (d & 7)))
- Virtual network interfaces can also be used to regenerate captured network traffic onto physical network segments for playback to downstream IDS appliances and network troubleshooting consoles.
- Figure 36 depicts a Virtual Interface mapped to a specific shot chain.
- Virtual Network interfaces also can employ a filter bit table during regeneration to filter out network packets that do not conform with specific include/exclude mask criteria.
- Virtual Network interfaces can be configured to regenerate network traffic at full physical network line rates or at the rates and UTC/microsecond timing the network packets were captured.
- Time replay virtual network interfaces (ift#) are employed to replay captured traffic to downstream devices that need to receive traffic at the original capture timing.
- Raw Virtual Network Interfaces (ifp#) will replay captured and filtered content at the full line supported by the physical interface.
- Regeneration creates a unique process for each regenerated virtual network interface to physical interface session. This process reads from the virtual network device and outputs the data to the physical interface upon each return from a request to read a slot chain. A PJHANDLE context is maintained for each unique regeneration session with a unique view into the captured slot chain being read.
- the regeneration process con be configured to limit data output on a physical segment in 1 mb/s (megabit per second) increments.
- the current embodiment of the invention allows these increments to span 1-10000 mb/s configurable per regeneration thread.
- Regeneration steps consist of mapping a PJHANDLE context to a virtual interface adapter and reading packets from an active slot chain until the interface reaches the end of the slot chain and blocks until more packet traffic arrives.
- packets are read from the slot chain, they are formatted into system dependent transmission units (skb's on Linux) and queued for transmission on a target physical network interface.
- the regeneration algorithm meters the total bytes transmitted over a target physical interface relative to the defined value for maximum bytes per second set by the user space application that initiated a regeneration process.
- the current embodiment of packet and protocol regeneration is instrumented as a polled method rather than event driven method.
- the regeneration algorithm is more fully described as:
- VIRTUAL_SETUP *v (VIRTUAL_SETUP *)arg
- dev dev_get_by_index(v->pindex); if (Idev) return 0;
- skb create_xmit_packet(v->pindex, &err, &skb_len); if (Iskb)
- pindex regen_chain_packet(v->interface, skb, skbjen, p_handle, &length, NULL, NULL,
- dev dev_get_by_index(v->pindex); if (!dev) return 0;
- dev->tx_queue_len tx_queue_len; dev_put(dev);
- the primary capture (type 0x97) disk space record for a DSFS system can be configured to map to multiple Archive Storage (type 0x98) partitions in an FC-AL clustered fiber channel System Area Network.
- Figure 37 depicts the DSFS primary capture node mapped onto multiple archive storage partitions in FC-AL Raid Array.
- active slot LRU slot cache elements can be mirrored to flush in parallel to a remote pool of slot storage as well as the primary disk record store.
- This architecture allows large pools of cache storage to be instrumented over a SAN fiber channel network with the primary capture partition serving as a tiered cache that replicates captured slots into long term network storage.
- the DSFS also supports user-space replicating file systems such as Intermezzo, Coda, Unison and rsync of 0x97 type partitions to 0X98 partitions as is known in the art.
- This architecture allows days, week, months, or even years of network packet data to be archived and indexed for off line post analysis operations, auditing, and network transaction accounting purposes.
- Primary Capture partitions contain a table of mapped archive partitions that may be used to allocate slot storage. As slots are allocated and pinned by adapters and subsequently filled, if a particular primary storage partition has an associated map of archive storage partitions, the primary capture partitions creates dual I/O links into the archive storage and initiates a mirrored write of a particular slot to both the primary capture partition and the archive storage partition in tandem. Slot chains located on archive storage partitions only export two primary slot chains. The VFS dynamic presents the slots in a replica chain (chain 0) and an archive chain(1). As slots are allocated from an Archive Storage partition, they are linked into the replica partition.
- Originating interface name, MAC address, and machine host name are also annotated in the additional tables present on a type 0x98 partition to identify the source name of the machine and interface information relative to a particular slot.
- Altering the attributes by setting an slot to read-only on an archive partition moves the slot from the replica slot chain (0) to the permanent archive slot chain (1).
- Slot allocation for selection of eligible targets for slot recycle on archive storage partitions is always biased to use the replica chain for slot reclamation. Slots stored on the archive slot chain (1) are only recycled if all slots in a given archive storage partition replica chain (0) have been converted to entries on the archive slot chain (1). In both cases, the oldest slots are targeted for recycle when an archive storage partition becomes fully populated. This allows forensic investigators the ability to pin specific slots of interest in an archive chain for permanent archival.
- the slot bitmap table records a value of 0 for any slots that have not been mirrored due to system unavailability, and a background re- mirroring process is spawned when the off line storage becomes active and re-mirrors the slot cache elements onto the target archive storage partitions with a background process.
- the system can also be configured to simply drop captured slots on the primary capture partition and not attempt mirroring of slots lost during an off line storage event for a group of archive partitions.
- FIG. 39 depicts mirroring of captured data in a SAN (System Area Network
- Slot allocation for SAN attached storage arrays that host archive storage partitions can be configured to allow stripe allocation of slots or contiguous slot allocation for a particular disk space record primary capture partition.
- Stripe allocation allows the primary capture partition to round robin a slot allocation for each entry in the primary capture map of archive storage partitions mapped to a primary capture partition. This allows distributed writes to be striped at a slot granularity across several remote fiber channel arrays in parallel and provides increased write performance.
- Contiguous allocation hard maps primary capture partitions to archive storage partitions in a linear fashion. Off line indexing is supported by tagging each captured packet with a globally unique identifier that allows rapid searching and retrieval on a per packet basis of capture network packets.
- Figure 40 depicts the method for tagging captured packets. These indexes are built during capture and combine the source MAC address of the capturing network adapter, the slot address and packet index within a slot, and protocol and layer 3 address information. These indexes are exposed through the /index subdirectory in the virtual file system per slot and are stored in 64K allocation clusters that are chained from the Slot Header located in the slot cache element.
- Off line indexes allow external applications to import indexing information for captured network traffic into off line databases and allow rapid search and retrieval of captured network packets through user space P_HANDLE context pointers.
- the globally unique identifier is guaranteed to be unique since it incorporates the unique MAC address of the network adapter that captured the packet payload.
- the global packet identifier also stores Ipv4 and Ipv6 address information per packet and supports Ipv4 and Ipv ⁇ indexing.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Library & Information Science (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Communication Control (AREA)
- Telephonic Communication Services (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims
Priority Applications (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2007548320A JP4759574B2 (en) | 2004-12-23 | 2005-12-16 | Method and apparatus for network packet capture distributed storage system |
US11/632,249 US7855974B2 (en) | 2004-12-23 | 2005-12-16 | Method and apparatus for network packet capture distributed storage system |
EP05854314.1A EP1832054B1 (en) | 2004-12-23 | 2005-12-16 | Method and apparatus for network packet capture distributed storage system |
AU2005322350A AU2005322350B2 (en) | 2004-12-23 | 2005-12-16 | Network packet capture distributed storage system |
CA2619141A CA2619141C (en) | 2004-12-23 | 2005-12-16 | Method and apparatus for network packet capture distributed storage system |
US12/416,276 US20090182953A1 (en) | 2004-12-23 | 2009-04-01 | Method and apparatus for network packet capture distributed storage system |
US12/469,744 US7684347B2 (en) | 2004-12-23 | 2009-05-21 | Method and apparatus for network packet capture distributed storage system |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US63870704P | 2004-12-23 | 2004-12-23 | |
US60/638,707 | 2004-12-23 |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US63224907A Continuation | 2004-12-23 | 2007-01-11 | |
US12/416,276 Continuation US20090182953A1 (en) | 2004-12-23 | 2009-04-01 | Method and apparatus for network packet capture distributed storage system |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2006071560A2 true WO2006071560A2 (en) | 2006-07-06 |
WO2006071560A3 WO2006071560A3 (en) | 2006-08-17 |
Family
ID=36615393
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2005/045566 WO2006071560A2 (en) | 2004-12-23 | 2005-12-16 | Network packet capture distributed storage system |
Country Status (6)
Country | Link |
---|---|
US (3) | US7855974B2 (en) |
EP (1) | EP1832054B1 (en) |
JP (2) | JP4759574B2 (en) |
AU (1) | AU2005322350B2 (en) |
CA (1) | CA2619141C (en) |
WO (1) | WO2006071560A2 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009021573A1 (en) * | 2007-08-14 | 2009-02-19 | Rohde & Schwarz Gmbh & Co. Kg | Method and device for logging communications connections at very high data rates |
WO2011060377A1 (en) | 2009-11-15 | 2011-05-19 | Solera Networks, Inc. | Method and apparatus for real time identification and recording of artifacts |
WO2011060368A1 (en) | 2009-11-15 | 2011-05-19 | Solera Networks, Inc. | Method and apparatus for storing and indexing high-speed network traffic data |
WO2013006185A1 (en) | 2011-07-06 | 2013-01-10 | Gigamon Llc | Network switch with traffic generation capability |
US9349024B2 (en) | 2011-01-18 | 2016-05-24 | International Business Machines Corporation | Assigning a data item to a storage location in a computing environment |
TWI711285B (en) * | 2019-09-18 | 2020-11-21 | 緯創資通股份有限公司 | Network failure detection method and network failure detection device |
Families Citing this family (130)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9026674B1 (en) * | 2010-03-22 | 2015-05-05 | Satish K Kanna | System and method for accurately displaying communications traffic information |
US7774604B2 (en) | 2003-12-10 | 2010-08-10 | Mcafee, Inc. | Verifying captured objects before presentation |
US7984175B2 (en) | 2003-12-10 | 2011-07-19 | Mcafee, Inc. | Method and apparatus for data capture and analysis system |
US7899828B2 (en) | 2003-12-10 | 2011-03-01 | Mcafee, Inc. | Tag data structure for maintaining relational data over captured objects |
US8656039B2 (en) | 2003-12-10 | 2014-02-18 | Mcafee, Inc. | Rule parser |
US8548170B2 (en) | 2003-12-10 | 2013-10-01 | Mcafee, Inc. | Document de-registration |
US7814327B2 (en) | 2003-12-10 | 2010-10-12 | Mcafee, Inc. | Document registration |
US7930540B2 (en) | 2004-01-22 | 2011-04-19 | Mcafee, Inc. | Cryptographic policy enforcement |
US7962591B2 (en) | 2004-06-23 | 2011-06-14 | Mcafee, Inc. | Object classification in a capture system |
US8560534B2 (en) | 2004-08-23 | 2013-10-15 | Mcafee, Inc. | Database for a capture system |
US7949849B2 (en) | 2004-08-24 | 2011-05-24 | Mcafee, Inc. | File system for a capture system |
EP1832054B1 (en) | 2004-12-23 | 2018-03-21 | Symantec Corporation | Method and apparatus for network packet capture distributed storage system |
US20100195538A1 (en) * | 2009-02-04 | 2010-08-05 | Merkey Jeffrey V | Method and apparatus for network packet capture distributed storage system |
US8351363B2 (en) * | 2005-04-08 | 2013-01-08 | Qualcomm Incorporated | Method and apparatus for enhanced file distribution in multicast or broadcast |
US7710969B2 (en) * | 2005-05-13 | 2010-05-04 | Texas Instruments Incorporated | Rapid I/O traffic system |
US7907608B2 (en) | 2005-08-12 | 2011-03-15 | Mcafee, Inc. | High speed packet capture |
US20070058620A1 (en) * | 2005-08-31 | 2007-03-15 | Mcdata Corporation | Management of a switch fabric through functionality conservation |
US7818326B2 (en) | 2005-08-31 | 2010-10-19 | Mcafee, Inc. | System and method for word indexing in a capture system and querying thereof |
US9143841B2 (en) | 2005-09-29 | 2015-09-22 | Brocade Communications Systems, Inc. | Federated management of intelligent service modules |
US7730011B1 (en) | 2005-10-19 | 2010-06-01 | Mcafee, Inc. | Attributes of captured objects in a capture system |
US7925971B2 (en) * | 2005-10-31 | 2011-04-12 | Solace Systems, Inc. | Transformation module for transforming documents from one format to other formats with pipelined processor having dedicated hardware resources |
US20070115916A1 (en) * | 2005-11-07 | 2007-05-24 | Samsung Electronics Co., Ltd. | Method and system for optimizing a network based on a performance knowledge base |
US7657104B2 (en) | 2005-11-21 | 2010-02-02 | Mcafee, Inc. | Identifying image type in a capture system |
US7953866B2 (en) | 2006-03-22 | 2011-05-31 | Mcdata Corporation | Protocols for connecting intelligent service modules in a storage area network |
US8504537B2 (en) | 2006-03-24 | 2013-08-06 | Mcafee, Inc. | Signature distribution in a document registration system |
US20070255866A1 (en) * | 2006-05-01 | 2007-11-01 | Eliezer Aloni | Method and system for a user space TCP offload engine (TOE) |
US20070258443A1 (en) * | 2006-05-02 | 2007-11-08 | Mcdata Corporation | Switch hardware and architecture for a computer network |
US20070258380A1 (en) * | 2006-05-02 | 2007-11-08 | Mcdata Corporation | Fault detection, isolation and recovery for a switch system of a computer network |
US8010689B2 (en) | 2006-05-22 | 2011-08-30 | Mcafee, Inc. | Locational tagging in a capture system |
US7689614B2 (en) | 2006-05-22 | 2010-03-30 | Mcafee, Inc. | Query generation for a capture system |
US7958227B2 (en) | 2006-05-22 | 2011-06-07 | Mcafee, Inc. | Attributes of captured objects in a capture system |
US7916170B2 (en) * | 2006-06-07 | 2011-03-29 | Robert Charles Soltysik | CCTV pipeline inspection system data management system and computer-based monitoring/action application |
US7653006B1 (en) | 2007-03-12 | 2010-01-26 | Deja Vu Networks, Inc. | Network traffic capture and replay with transaction integrity and scaling |
US9961094B1 (en) | 2007-07-25 | 2018-05-01 | Xangati, Inc | Symptom detection using behavior probability density, network monitoring of multiple observation value types, and network monitoring using orthogonal profiling dimensions |
US7913046B2 (en) * | 2007-08-06 | 2011-03-22 | Dell Global B.V. - Singapore Branch | Method for performing a snapshot in a distributed shared file system |
US20090144388A1 (en) * | 2007-11-08 | 2009-06-04 | Rna Networks, Inc. | Network with distributed shared memory |
US20090225767A1 (en) * | 2008-03-05 | 2009-09-10 | Inventec Corporation | Network packet capturing method |
EP2324615B1 (en) * | 2008-05-21 | 2014-04-02 | McAfee, Inc. | System and method for discovery of network entities |
US8004998B2 (en) * | 2008-05-23 | 2011-08-23 | Solera Networks, Inc. | Capture and regeneration of a network data using a virtual software switch |
US8625642B2 (en) | 2008-05-23 | 2014-01-07 | Solera Networks, Inc. | Method and apparatus of network artifact indentification and extraction |
US8521732B2 (en) | 2008-05-23 | 2013-08-27 | Solera Networks, Inc. | Presentation of an extracted artifact based on an indexing technique |
US20090292736A1 (en) * | 2008-05-23 | 2009-11-26 | Matthew Scott Wood | On demand network activity reporting through a dynamic file system and method |
US8205242B2 (en) | 2008-07-10 | 2012-06-19 | Mcafee, Inc. | System and method for data mining and security policy management |
JP2010034721A (en) * | 2008-07-28 | 2010-02-12 | Fujitsu Ltd | Packet capture apparatus, packet capture method, and packet capture program |
US8271748B2 (en) * | 2008-08-12 | 2012-09-18 | Intel Corporation | Generating and/or receiving, at least one data access request |
US9253154B2 (en) | 2008-08-12 | 2016-02-02 | Mcafee, Inc. | Configuration management for a capture/registration system |
US8359402B2 (en) * | 2008-11-19 | 2013-01-22 | Seachange International, Inc. | Intercept device for providing content |
US8850591B2 (en) | 2009-01-13 | 2014-09-30 | Mcafee, Inc. | System and method for concept building |
US8706709B2 (en) | 2009-01-15 | 2014-04-22 | Mcafee, Inc. | System and method for intelligent term grouping |
US8473442B1 (en) | 2009-02-25 | 2013-06-25 | Mcafee, Inc. | System and method for intelligent state management |
US8447722B1 (en) | 2009-03-25 | 2013-05-21 | Mcafee, Inc. | System and method for data mining and security policy management |
US8667121B2 (en) | 2009-03-25 | 2014-03-04 | Mcafee, Inc. | System and method for managing data and policies |
KR20100107801A (en) * | 2009-03-26 | 2010-10-06 | 삼성전자주식회사 | Apparatus and method for antenna selection in wireless communication system |
US8769589B2 (en) * | 2009-03-31 | 2014-07-01 | At&T Intellectual Property I, L.P. | System and method to create a media content summary based on viewer annotations |
US8762334B1 (en) | 2009-04-29 | 2014-06-24 | Juniper Networks, Inc. | Distributed network anomaly detection |
US10992555B2 (en) | 2009-05-29 | 2021-04-27 | Virtual Instruments Worldwide, Inc. | Recording, replay, and sharing of live network monitoring views |
TWI389001B (en) * | 2009-06-01 | 2013-03-11 | Mstar Semiconductor Inc | File system and file system transforming method |
US8977705B2 (en) * | 2009-07-27 | 2015-03-10 | Verisign, Inc. | Method and system for data logging and analysis |
US8479300B2 (en) * | 2009-10-26 | 2013-07-02 | Delta Electronics, Inc. | Method for transmitting data and preventing unauthorized data duplication for human-machine interface device using mass storage class operating on universal serial bus |
US9501365B2 (en) * | 2009-12-28 | 2016-11-22 | Netapp, Inc. | Cloud-based disaster recovery of backup data and metadata |
US8472449B2 (en) * | 2010-03-02 | 2013-06-25 | Intrusion, Inc. | Packet file system |
US8448221B2 (en) * | 2010-03-12 | 2013-05-21 | Mcafee, Inc. | System, method, and computer program product for displaying network events in terms of objects managed by a security appliance and/or a routing device |
US20110296437A1 (en) * | 2010-05-28 | 2011-12-01 | Devendra Raut | Method and apparatus for lockless communication between cores in a multi-core processor |
WO2012015363A1 (en) * | 2010-07-30 | 2012-02-02 | Agency For Science, Technology And Research | Acquiring information from volatile memory of a mobile device |
US8352777B2 (en) * | 2010-10-04 | 2013-01-08 | Hewlett-Packard Development Company, L.P. | Replaying captured network traffic |
US8806615B2 (en) | 2010-11-04 | 2014-08-12 | Mcafee, Inc. | System and method for protecting specified data combinations |
US20120143824A1 (en) * | 2010-12-02 | 2012-06-07 | Microsoft Corporation | Protecting files that include editable metadata |
US9824091B2 (en) | 2010-12-03 | 2017-11-21 | Microsoft Technology Licensing, Llc | File system backup using change journal |
US8849991B2 (en) | 2010-12-15 | 2014-09-30 | Blue Coat Systems, Inc. | System and method for hypertext transfer protocol layered reconstruction |
US8620894B2 (en) | 2010-12-21 | 2013-12-31 | Microsoft Corporation | Searching files |
EP2659620B1 (en) * | 2010-12-29 | 2018-10-17 | Citrix Systems Inc. | Systems and methods for scalable n-core statistics aggregation |
US8666985B2 (en) | 2011-03-16 | 2014-03-04 | Solera Networks, Inc. | Hardware accelerated application-based pattern matching for real time classification and recording of network traffic |
US8996800B2 (en) | 2011-07-07 | 2015-03-31 | Atlantis Computing, Inc. | Deduplication of virtual machine files in a virtualized desktop environment |
US9229818B2 (en) | 2011-07-20 | 2016-01-05 | Microsoft Technology Licensing, Llc | Adaptive retention for backup data |
US9524243B1 (en) * | 2011-09-27 | 2016-12-20 | Emc Ip Holdng Company Llc | Scalable monolithic data storage system for cloud environment |
US10318426B1 (en) * | 2011-09-27 | 2019-06-11 | EMC IP Holding Company LLC | Cloud capable storage platform with computation operating environment for storage and generic applications |
US8700691B2 (en) | 2011-12-05 | 2014-04-15 | Microsoft Corporation | Minimal download and simulated page navigation features |
US20130246431A1 (en) | 2011-12-27 | 2013-09-19 | Mcafee, Inc. | System and method for providing data protection workflows in a network environment |
US9846605B2 (en) | 2012-01-19 | 2017-12-19 | Microsoft Technology Licensing, Llc | Server-side minimal download and error failover |
US10289743B2 (en) | 2012-01-19 | 2019-05-14 | Microsoft Technology Licensing, Llc | Client-side minimal download and simulated page navigation features |
US8655769B2 (en) * | 2012-03-16 | 2014-02-18 | Cape City Command, Llc | Method and system for improving equity trade order acknowledgement times |
EP2717515A1 (en) * | 2012-06-30 | 2014-04-09 | Huawei Technologies Co., Ltd. | Virtual port monitoring method and device |
US9852073B2 (en) | 2012-08-07 | 2017-12-26 | Dell Products L.P. | System and method for data redundancy within a cache |
US8850182B1 (en) | 2012-09-28 | 2014-09-30 | Shoretel, Inc. | Data capture for secure protocols |
US20140101761A1 (en) * | 2012-10-09 | 2014-04-10 | James Harlacher | Systems and methods for capturing, replaying, or analyzing time-series data |
US9588874B2 (en) * | 2012-12-14 | 2017-03-07 | Microsoft Technology Licensing, Llc | Remote device automation using a device services bridge |
US9069472B2 (en) * | 2012-12-21 | 2015-06-30 | Atlantis Computing, Inc. | Method for dispersing and collating I/O's from virtual machines for parallelization of I/O access and redundancy of storing virtual machine data |
US9277010B2 (en) | 2012-12-21 | 2016-03-01 | Atlantis Computing, Inc. | Systems and apparatuses for aggregating nodes to form an aggregated virtual storage for a virtualized desktop environment |
US10305760B2 (en) | 2013-01-03 | 2019-05-28 | Entit Software Llc | Identifying an analysis reporting message in network traffic |
US9250946B2 (en) | 2013-02-12 | 2016-02-02 | Atlantis Computing, Inc. | Efficient provisioning of cloned virtual machine images using deduplication metadata |
US9372865B2 (en) | 2013-02-12 | 2016-06-21 | Atlantis Computing, Inc. | Deduplication metadata access in deduplication file system |
US9471590B2 (en) | 2013-02-12 | 2016-10-18 | Atlantis Computing, Inc. | Method and apparatus for replicating virtual machine images using deduplication metadata |
US9558199B2 (en) * | 2013-03-07 | 2017-01-31 | Jive Software, Inc. | Efficient data deduplication |
US10033609B1 (en) | 2013-05-07 | 2018-07-24 | Ca, Inc. | Low impact passive monitoring of application performance |
US20180081749A1 (en) * | 2013-12-04 | 2018-03-22 | International Business Machines Corporation | Performance ranking of read requests in a distributed storage network |
US9325639B2 (en) | 2013-12-17 | 2016-04-26 | At&T Intellectual Property I, L.P. | Hierarchical caching system for lossless network packet capture applications |
US20150341244A1 (en) * | 2014-05-22 | 2015-11-26 | Virtual Instruments Corporation | Performance Analysis of a Time-Varying Network |
US10250470B1 (en) | 2014-08-24 | 2019-04-02 | Virtual Instruments Worldwide | Push pull data collection |
US10009237B1 (en) | 2014-08-24 | 2018-06-26 | Virtual Instruments Worldwide | Cross silo time stiching |
US10079740B2 (en) | 2014-11-04 | 2018-09-18 | Fermi Research Alliance, Llc | Packet capture engine for commodity network interface cards in high-speed networks |
US10185830B1 (en) * | 2014-12-31 | 2019-01-22 | EMC IP Holding Company LLC | Big data analytics in a converged infrastructure system |
US9646350B1 (en) * | 2015-01-14 | 2017-05-09 | Amdocs Software Systems Limited | System, method, and computer program for performing operations on network files including captured billing event information |
US9703661B2 (en) | 2015-02-05 | 2017-07-11 | International Business Machines Corporation | Eliminate corrupted portions of cache during runtime |
US10691661B2 (en) | 2015-06-03 | 2020-06-23 | Xilinx, Inc. | System and method for managing the storing of data |
US10733167B2 (en) * | 2015-06-03 | 2020-08-04 | Xilinx, Inc. | System and method for capturing data to provide to a data analyser |
US10057142B2 (en) * | 2015-08-19 | 2018-08-21 | Microsoft Technology Licensing, Llc | Diagnostic framework in computing systems |
US9935858B1 (en) | 2015-08-24 | 2018-04-03 | Xangati, Inc | Enhanched flow processing |
CN107102695B (en) * | 2016-02-22 | 2020-07-24 | 佛山市顺德区顺达电脑厂有限公司 | Method for determining mounting position of abnormal hard disk for cluster storage system |
FI127335B (en) | 2016-05-27 | 2018-04-13 | Cysec Ice Wall Oy | Logging of data traffic in a computer network |
US9892622B2 (en) | 2016-05-27 | 2018-02-13 | At&T Intellectual Property I, L.P. | Emergency event virtual network function deployment and configuration |
US9965211B2 (en) | 2016-09-08 | 2018-05-08 | Cisco Technology, Inc. | Dynamic packet buffers with consolidation of low utilized memory banks |
JP6839347B2 (en) * | 2016-11-02 | 2021-03-10 | 富士通株式会社 | Packet capture program, packet capture device and packet capture method |
CN108337181B (en) * | 2017-01-20 | 2021-05-28 | 深圳市中兴微电子技术有限公司 | Method and device for managing congestion of switching network |
US10715433B2 (en) | 2017-03-31 | 2020-07-14 | Mitsubishi Electric Corporation | Information processing apparatus and information processing method |
US10437729B2 (en) | 2017-04-19 | 2019-10-08 | International Business Machines Corporation | Non-disruptive clearing of varying address ranges from cache |
JP7003562B2 (en) | 2017-10-16 | 2022-01-20 | 富士通株式会社 | Miller packet control program, mirror packet control method, and mirror packet control device |
US11416616B2 (en) | 2017-11-30 | 2022-08-16 | Forcepoint Llc | Secure boot chain for live boot systems |
US10838763B2 (en) * | 2018-07-17 | 2020-11-17 | Xilinx, Inc. | Network interface device and host processing device |
US10887251B2 (en) * | 2018-09-13 | 2021-01-05 | International Business Machines Corporation | Fault-tolerant architecture for packet capture |
CN111104047B (en) * | 2018-10-25 | 2023-08-25 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer readable storage medium for managing redundant array of disks |
KR102006475B1 (en) * | 2019-01-18 | 2019-08-01 | 넷마블 주식회사 | Method and apparatus for detecting intrusion |
US11080160B1 (en) | 2019-01-29 | 2021-08-03 | Virtual Instruments Worldwide, Inc. | Self-learning and best-practice profiling and alerting with relative and absolute capacity |
US11042483B2 (en) | 2019-04-26 | 2021-06-22 | International Business Machines Corporation | Efficient eviction of whole set associated cache or selected range of addresses |
KR20210016938A (en) * | 2019-08-06 | 2021-02-17 | 에스케이하이닉스 주식회사 | Data processing system and operating method thereof |
CN110647548B (en) * | 2019-09-23 | 2023-03-21 | 浪潮软件股份有限公司 | Method and system for converting streaming data into batch based on NiFi and state value thereof |
US11842057B2 (en) | 2019-12-09 | 2023-12-12 | Dell Products L.P. | Seamless creation of raid arrays with optimized boot time |
CN114070900B (en) * | 2020-07-27 | 2023-04-07 | 大唐移动通信设备有限公司 | DPDK-based packet capture processing method and device |
CN111953568B (en) * | 2020-08-19 | 2022-04-08 | 杭州迪普科技股份有限公司 | Method and device for managing packet loss information |
CN115604207B (en) * | 2022-12-12 | 2023-03-10 | 成都数默科技有限公司 | Session-oriented network flow storage and indexing method |
CN116431684B (en) * | 2023-04-18 | 2024-03-19 | 中船海神医疗科技有限公司 | Diagnosis and treatment data storage and playback method and system for portable life support system |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS59127453A (en) | 1983-01-10 | 1984-07-23 | Nec Corp | Line monitoring device |
US20030135525A1 (en) | 2001-07-17 | 2003-07-17 | Huntington Stephen Glen | Sliding window packet management systems |
Family Cites Families (192)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US13222A (en) * | 1855-07-10 | Washing-machine | ||
US175167A (en) * | 1876-03-21 | Improvement in vehicle-seat locks | ||
US28169A (en) * | 1860-05-08 | Improvement in plows | ||
US113217A (en) * | 1871-03-28 | Improvement in saw-mills | ||
US73895A (en) * | 1868-01-28 | Benjamin illingworth | ||
US88040A (en) * | 1869-03-23 | Improvement in mechanical movement | ||
US28161A (en) * | 1860-05-08 | Machines | ||
US116470A (en) * | 1871-06-27 | Improvement in machinery for making roofing-felt | ||
US89937A (en) * | 1869-05-11 | Improvement in machine for cutting- mouldings in wood | ||
US162971A (en) * | 1875-05-04 | Improvement in processes for manufacture of sole-fastenings | ||
US122801A (en) * | 1872-01-16 | Improvement in harvester-cutters | ||
US291755A (en) * | 1884-01-08 | Straw-stacker | ||
US116403A (en) * | 1871-06-27 | Improvement in steam-traps | ||
US92057A (en) * | 1869-06-29 | Improvement in hydraotiic presses | ||
US103531A (en) * | 1870-05-24 | Improvement in machines for putting together and tiring wheels | ||
US85507A (en) * | 1869-01-05 | John d | ||
US291757A (en) * | 1884-01-08 | Corn-planter | ||
US165052A (en) * | 1875-06-29 | Improvement in car-starters | ||
US140295A (en) * | 1873-06-24 | Improvement in tassel-clips for curtains | ||
US147263A (en) * | 1874-02-10 | Improvement in burial-caskets | ||
US165009A (en) * | 1875-06-29 | Improvement in pencil-cases | ||
US5860136A (en) | 1989-06-16 | 1999-01-12 | Fenner; Peter R. | Method and apparatus for use of associated memory with large key spaces |
JP2563603B2 (en) * | 1989-09-11 | 1996-12-11 | 松下電器産業株式会社 | Data monitoring device |
US5440719A (en) | 1992-10-27 | 1995-08-08 | Cadence Design Systems, Inc. | Method simulating data traffic on network in accordance with a client/sewer paradigm |
US5274643A (en) | 1992-12-11 | 1993-12-28 | Stratacom, Inc. | Method for optimizing a network having virtual circuit routing over virtual paths |
JP2959943B2 (en) * | 1993-12-10 | 1999-10-06 | アンリツ株式会社 | Information retrieval device for network monitoring system |
US5526283A (en) * | 1994-01-26 | 1996-06-11 | International Business Machines Corporation | Realtime high speed data capture in response to an event |
EP0702473A1 (en) * | 1994-09-19 | 1996-03-20 | International Business Machines Corporation | A method and an apparatus for shaping the output traffic in a fixed length cell switching network node |
US5758178A (en) * | 1996-03-01 | 1998-05-26 | Hewlett-Packard Company | Miss tracking system and method |
US6400681B1 (en) * | 1996-06-20 | 2002-06-04 | Cisco Technology, Inc. | Method and system for minimizing the connection set up time in high speed packet switching networks |
US6108637A (en) * | 1996-09-03 | 2000-08-22 | Nielsen Media Research, Inc. | Content display monitor |
US6101543A (en) | 1996-10-25 | 2000-08-08 | Digital Equipment Corporation | Pseudo network adapter for frame capture, encapsulation and encryption |
US6453345B2 (en) * | 1996-11-06 | 2002-09-17 | Datadirect Networks, Inc. | Network security and surveillance system |
US6643696B2 (en) * | 1997-03-21 | 2003-11-04 | Owen Davis | Method and apparatus for tracking client interaction with a network resource and creating client profiles and resource database |
SE9702688D0 (en) | 1997-07-11 | 1997-07-11 | Ericsson Telefon Ab L M | A method and system for interconnicting ring networks |
US6145108A (en) | 1997-09-04 | 2000-11-07 | Conexant Systems, Inc. | Retransmission packet capture system within a wireless multiservice communications environment |
US6041053A (en) * | 1997-09-18 | 2000-03-21 | Microsfot Corporation | Technique for efficiently classifying packets using a trie-indexed hierarchy forest that accommodates wildcards |
US5956721A (en) * | 1997-09-19 | 1999-09-21 | Microsoft Corporation | Method and computer program product for classifying network communication packets processed in a network stack |
US6434620B1 (en) * | 1998-08-27 | 2002-08-13 | Alacritech, Inc. | TCP/IP offload network interface device |
US7032242B1 (en) * | 1998-03-05 | 2006-04-18 | 3Com Corporation | Method and system for distributed network address translation with network security features |
US7450560B1 (en) | 1998-03-05 | 2008-11-11 | 3Com Corporation | Method for address mapping in a network access system and a network access device for use therewith |
US20070050465A1 (en) | 1998-03-19 | 2007-03-01 | Canter James M | Packet capture agent for use in field assets employing shared bus architecture |
GB2337903B (en) | 1998-05-28 | 2000-06-07 | 3Com Corp | Methods and apparatus for collecting storing processing and using network traffic data |
US7154896B1 (en) | 1998-07-24 | 2006-12-26 | Ericsson, Ab | ATM traffic having unknown characteristics including traffic with weighted priorities and traffic without weighted priorities |
US6675218B1 (en) * | 1998-08-14 | 2004-01-06 | 3Com Corporation | System for user-space network packet modification |
US6628652B1 (en) | 1998-09-18 | 2003-09-30 | Lucent Technologies Inc. | Flexible telecommunications switching network |
US6807667B1 (en) * | 1998-09-21 | 2004-10-19 | Microsoft Corporation | Method and system of an application program interface for abstracting network traffic control components to application programs |
US6370622B1 (en) * | 1998-11-20 | 2002-04-09 | Massachusetts Institute Of Technology | Method and apparatus for curious and column caching |
JPH11288387A (en) * | 1998-12-11 | 1999-10-19 | Fujitsu Ltd | Disk cache device |
AU768572B2 (en) | 1999-02-25 | 2003-12-18 | Nippon Telegraph & Telephone Corporation | Traffic monitoring equipment and system and method for datagram transfer |
US6628617B1 (en) | 1999-03-03 | 2003-09-30 | Lucent Technologies Inc. | Technique for internetworking traffic on connectionless and connection-oriented networks |
US6754202B1 (en) * | 1999-06-15 | 2004-06-22 | Altigen Communications, Inc. | Network call-back data capture method and apparatus |
US6578084B1 (en) * | 1999-10-15 | 2003-06-10 | Cisco Technology, Inc. | Packet processing using encapsulation and decapsulation chains |
WO2001047162A1 (en) | 1999-12-23 | 2001-06-28 | Cetacean Networks, Inc. | Network switch with packet scheduling |
US6693909B1 (en) * | 2000-05-05 | 2004-02-17 | Fujitsu Network Communications, Inc. | Method and system for transporting traffic in a packet-switched network |
US7075927B2 (en) * | 2000-05-05 | 2006-07-11 | Fujitsu Limited | Method and system for quality of service (QoS) support in a packet-switched network |
US6789125B1 (en) | 2000-05-10 | 2004-09-07 | Cisco Technology, Inc. | Distributed network traffic load balancing technique implemented without gateway router |
US7162649B1 (en) * | 2000-06-30 | 2007-01-09 | Internet Security Systems, Inc. | Method and apparatus for network assessment and authentication |
JP2002026935A (en) * | 2000-07-11 | 2002-01-25 | Lac Co Ltd | Frame monitoring device and storage medium |
US7058015B1 (en) * | 2000-08-04 | 2006-06-06 | Arbor Networks, Inc. | Distributed solution for regulating network traffic |
JP3474155B2 (en) | 2000-08-17 | 2003-12-08 | 日立電子サービス株式会社 | Network system and network monitoring method |
JP2002101128A (en) | 2000-09-26 | 2002-04-05 | Kddi Corp | Traffic generating device |
US6522629B1 (en) * | 2000-10-10 | 2003-02-18 | Tellicent Inc. | Traffic manager, gateway signaling and provisioning service for all packetized networks with total system-wide standards for broad-band applications including all legacy services |
US7027412B2 (en) * | 2000-11-10 | 2006-04-11 | Veritas Operating Corporation | System for dynamic provisioning of secure, scalable, and extensible networked computer environments |
US20020089937A1 (en) | 2000-11-16 | 2002-07-11 | Srinivasan Venkatachary | Packet matching method and system |
US7002926B1 (en) * | 2000-11-30 | 2006-02-21 | Western Digital Ventures, Inc. | Isochronous switched fabric network |
KR100379336B1 (en) * | 2000-12-01 | 2003-04-10 | 주식회사 하이닉스반도체 | Fabrication method of isolation region for semiconductor devices |
US7218632B1 (en) * | 2000-12-06 | 2007-05-15 | Cisco Technology, Inc. | Packet processing engine architecture |
EP1354300B1 (en) * | 2000-12-19 | 2007-08-01 | Azoteq (PTY) Limited | Method of and apparatus for transferring data |
US7130466B2 (en) * | 2000-12-21 | 2006-10-31 | Cobion Ag | System and method for compiling images from a database and comparing the compiled images with known images |
US20020085507A1 (en) | 2000-12-28 | 2002-07-04 | Maple Optical Systems, Inc. | Address learning technique in a data communication network |
US6907520B2 (en) * | 2001-01-11 | 2005-06-14 | Sun Microsystems, Inc. | Threshold-based load address prediction and new thread identification in a multithreaded microprocessor |
US7061874B2 (en) * | 2001-01-26 | 2006-06-13 | Broadcom Corporation | Method, system and computer program product for classifying packet flows with a bit mask |
US20020147803A1 (en) * | 2001-01-31 | 2002-10-10 | Dodd Timothy David | Method and system for calculating risk in association with a security audit of a computer network |
US6516380B2 (en) * | 2001-02-05 | 2003-02-04 | International Business Machines Corporation | System and method for a log-based non-volatile write cache in a storage controller |
US6999454B1 (en) * | 2001-02-09 | 2006-02-14 | Nortel Networks Limited | Information routing system and apparatus |
US7120129B2 (en) * | 2001-03-13 | 2006-10-10 | Microsoft Corporation | System and method for achieving zero-configuration wireless computing and computing device incorporating same |
US6993037B2 (en) * | 2001-03-21 | 2006-01-31 | International Business Machines Corporation | System and method for virtual private network network address translation propagation over nested connections with coincident local endpoints |
US7526795B2 (en) * | 2001-03-27 | 2009-04-28 | Micron Technology, Inc. | Data security for digital data storage |
US7009979B1 (en) | 2001-03-30 | 2006-03-07 | Agere Systems Inc. | Virtual segmentation system and method of operation thereof |
US7024609B2 (en) * | 2001-04-20 | 2006-04-04 | Kencast, Inc. | System for protecting the transmission of live data streams, and upon reception, for reconstructing the live data streams and recording them into files |
US6928471B2 (en) * | 2001-05-07 | 2005-08-09 | Quest Software, Inc. | Method and apparatus for measurement, analysis, and optimization of content delivery |
US7065482B2 (en) * | 2001-05-17 | 2006-06-20 | International Business Machines Corporation | Internet traffic analysis tool |
US6744739B2 (en) | 2001-05-18 | 2004-06-01 | Micromuse Inc. | Method and system for determining network characteristics using routing protocols |
US7237264B1 (en) * | 2001-06-04 | 2007-06-26 | Internet Security Systems, Inc. | System and method for preventing network misuse |
US7126944B2 (en) | 2001-07-05 | 2006-10-24 | Intel Corporation | Routing packets across multiple forwarding elements |
US6958998B2 (en) | 2001-07-09 | 2005-10-25 | International Business Machines Corporation | Traffic management in packet-based networks |
US7047297B2 (en) * | 2001-07-17 | 2006-05-16 | Mcafee, Inc. | Hierarchically organizing network data collected from full time recording machines and efficiently filtering the same |
US7200122B2 (en) * | 2001-09-06 | 2007-04-03 | Avaya Technology Corp. | Using link state information to discover IP network topology |
US7444679B2 (en) | 2001-10-31 | 2008-10-28 | Hewlett-Packard Development Company, L.P. | Network, method and computer readable medium for distributing security updates to select nodes on a network |
US7126954B2 (en) | 2001-11-13 | 2006-10-24 | General Instrument Corporation | Virtual gateway |
US6782444B1 (en) * | 2001-11-15 | 2004-08-24 | Emc Corporation | Digital data storage subsystem including directory for efficiently providing formatting information for stored records |
US7283478B2 (en) | 2001-11-28 | 2007-10-16 | Corrigent Systems Ltd. | Traffic engineering in bi-directional ring networks |
US7203173B2 (en) * | 2002-01-25 | 2007-04-10 | Architecture Technology Corp. | Distributed packet capture and aggregation |
US7376731B2 (en) * | 2002-01-29 | 2008-05-20 | Acme Packet, Inc. | System and method for providing statistics gathering within a packet network |
US7529242B1 (en) * | 2002-02-15 | 2009-05-05 | Symantec Corporation | Routing network packets for multi-processor network flow analysis |
JP3750803B2 (en) * | 2002-03-29 | 2006-03-01 | 横河電機株式会社 | Packet log recording device |
US7277399B1 (en) | 2002-04-08 | 2007-10-02 | Cisco Technology, Inc. | Hardware-based route cache using prefix length |
US7116643B2 (en) | 2002-04-30 | 2006-10-03 | Motorola, Inc. | Method and system for data in a collection and route discovery communication network |
JP4032816B2 (en) | 2002-05-08 | 2008-01-16 | 株式会社日立製作所 | Storage network topology management system |
CA2387654A1 (en) * | 2002-05-24 | 2003-11-24 | Alcatel Canada Inc. | Partitioned interface architecture for transmission of broadband network traffic to and from an access network |
US7177311B1 (en) * | 2002-06-04 | 2007-02-13 | Fortinet, Inc. | System and method for routing traffic through a virtual router-based network switch |
US20030231632A1 (en) | 2002-06-13 | 2003-12-18 | International Business Machines Corporation | Method and system for packet-level routing |
US20060013222A1 (en) | 2002-06-28 | 2006-01-19 | Brocade Communications Systems, Inc. | Apparatus and method for internet protocol data processing in a storage processing device |
JP2004048267A (en) * | 2002-07-10 | 2004-02-12 | Sharp Corp | Signature method for preventing falsification of rewritable media, signature apparatus for executing method thereof, signature system for preventing falsification provided with the apparatus, signature program for preventing falsification to realize method thereof, and computer-readable recording medium with the falsification preventing signature program recorded thereon |
US7254562B2 (en) | 2002-07-11 | 2007-08-07 | Hewlett-Packard Development Company, L.P. | Rule-based packet selection, storage, and access method and system |
US7039018B2 (en) * | 2002-07-17 | 2006-05-02 | Intel Corporation | Technique to improve network routing using best-match and exact-match techniques |
EP1387527A1 (en) | 2002-07-30 | 2004-02-04 | Agilent Technologies Inc. | Identifying network routers and paths |
WO2004014001A1 (en) * | 2002-08-02 | 2004-02-12 | Nms Communications | Methods and apparatus for network signal aggregation and bandwidth reduction |
KR20050032588A (en) * | 2002-08-08 | 2005-04-07 | 마츠시타 덴끼 산교 가부시키가이샤 | Encrypting/decrypting device and method, encrypting device and method, decrypting device and method, and transmitting/receiving device |
US7440464B2 (en) | 2002-08-29 | 2008-10-21 | Nokia Corporation | Server control plane connections recovery in a server-gateway architecture based telecommunication network |
US7529276B1 (en) * | 2002-09-03 | 2009-05-05 | Cisco Technology, Inc. | Combined jitter and multiplexing systems and methods |
US7457277B1 (en) | 2002-09-20 | 2008-11-25 | Mahi Networks, Inc. | System and method for network layer protocol routing in a peer model integrated optical network |
US7533256B2 (en) * | 2002-10-31 | 2009-05-12 | Brocade Communications Systems, Inc. | Method and apparatus for encryption of data on storage units using devices inside a storage area network fabric |
GB0226249D0 (en) * | 2002-11-11 | 2002-12-18 | Clearspeed Technology Ltd | Traffic handling system |
US7266120B2 (en) | 2002-11-18 | 2007-09-04 | Fortinet, Inc. | System and method for hardware accelerated packet multicast in a virtual routing system |
US7359930B2 (en) * | 2002-11-21 | 2008-04-15 | Arbor Networks | System and method for managing computer networks |
US7433326B2 (en) | 2002-11-27 | 2008-10-07 | Cisco Technology, Inc. | Methods and devices for exchanging peer parameters between network devices |
US7376969B1 (en) * | 2002-12-02 | 2008-05-20 | Arcsight, Inc. | Real time monitoring and analysis of events from multiple network security devices |
GB0304807D0 (en) * | 2003-03-03 | 2003-04-09 | Cambridge Internetworking Ltd | Data protocol |
WO2004080026A1 (en) | 2003-03-04 | 2004-09-16 | Lukas Wunner | Method, system and storage medium for introducing data network accessibility information |
US7441267B1 (en) | 2003-03-19 | 2008-10-21 | Bbn Technologies Corp. | Method and apparatus for controlling the flow of data across a network interface |
US7525963B2 (en) * | 2003-04-24 | 2009-04-28 | Microsoft Corporation | Bridging subnet broadcasts across subnet boundaries |
US7240166B2 (en) * | 2003-05-01 | 2007-07-03 | International Business Machines Corporation | Method and apparatus for implementing packet work area accesses and buffer sharing |
US7522613B2 (en) * | 2003-05-07 | 2009-04-21 | Nokia Corporation | Multiplexing media components of different sessions |
US7391769B2 (en) * | 2003-06-27 | 2008-06-24 | Lucent Technologies Inc. | Packet aggregation for real time services on packet data networks |
JP4418286B2 (en) * | 2003-07-14 | 2010-02-17 | 富士通株式会社 | Distributed storage system |
US7525910B2 (en) * | 2003-07-16 | 2009-04-28 | Qlogic, Corporation | Method and system for non-disruptive data capture in networks |
US7522594B2 (en) * | 2003-08-19 | 2009-04-21 | Eye Ball Networks, Inc. | Method and apparatus to permit data transmission to traverse firewalls |
US7467202B2 (en) | 2003-09-10 | 2008-12-16 | Fidelis Security Systems | High-performance network content analysis platform |
JP3947146B2 (en) * | 2003-09-18 | 2007-07-18 | 富士通株式会社 | Routing loop detection program and routing loop detection method |
JP4199772B2 (en) | 2003-09-25 | 2008-12-17 | 富士通株式会社 | Recording / reproducing method of optical recording medium |
US6956820B2 (en) | 2003-10-01 | 2005-10-18 | Santera Systems, Inc. | Methods, systems, and computer program products for voice over IP (VoIP) traffic engineering and path resilience using network-aware media gateway |
US7512078B2 (en) * | 2003-10-15 | 2009-03-31 | Texas Instruments Incorporated | Flexible ethernet bridge |
US7408938B1 (en) | 2003-10-15 | 2008-08-05 | Microsoft Coporation | System and method for efficient broadcast of information over a network |
CA2545496C (en) * | 2003-11-11 | 2012-10-30 | Citrix Gateways, Inc. | Virtual private network with pseudo server |
KR100567320B1 (en) | 2003-11-28 | 2006-04-04 | 한국전자통신연구원 | Flow generation method for Internet traffic measurement |
US20070297349A1 (en) | 2003-11-28 | 2007-12-27 | Ofir Arkin | Method and System for Collecting Information Relating to a Communication Network |
US7984175B2 (en) | 2003-12-10 | 2011-07-19 | Mcafee, Inc. | Method and apparatus for data capture and analysis system |
CN100349408C (en) | 2004-02-12 | 2007-11-14 | 华为技术有限公司 | Method for realizing configuration data real-time synchronization for network management system and network element device |
US7457870B1 (en) | 2004-02-27 | 2008-11-25 | Packeteer, Inc. | Methods, apparatuses and systems facilitating classification of web services network traffic |
US7881319B2 (en) | 2004-02-27 | 2011-02-01 | Actix Limited | Data storage and processing systems |
JP4323987B2 (en) | 2004-03-16 | 2009-09-02 | キヤノン株式会社 | Network switch and packet relay method for relaying packets while maintaining the real-time property of packets |
US7532623B2 (en) * | 2004-03-24 | 2009-05-12 | Bbn Technologies Corp. | Methods for wireless mesh multicasting |
US7292591B2 (en) | 2004-03-30 | 2007-11-06 | Extreme Networks, Inc. | Packet processing system architecture and method |
US7480255B2 (en) * | 2004-05-27 | 2009-01-20 | Cisco Technology, Inc. | Data structure identifying for multiple addresses the reverse path forwarding information for a common intermediate node and its use |
JP2008507928A (en) | 2004-07-23 | 2008-03-13 | サイトリックス システムズ, インコーポレイテッド | System and method for optimizing communication between network nodes |
US7519010B1 (en) * | 2004-08-30 | 2009-04-14 | Juniper Networks, Inc. | Inter-autonomous system (AS) multicast virtual private networks |
US7489635B2 (en) * | 2004-09-24 | 2009-02-10 | Lockheed Martin Corporation | Routing cost based network congestion control for quality of service |
US7840725B2 (en) | 2004-09-28 | 2010-11-23 | Hewlett-Packard Development Company, L.P. | Capture of data in a computer network |
US7457296B2 (en) | 2004-09-30 | 2008-11-25 | Intel Corporation | Method and apparatus for sorting packets in packet schedulers using a connected trie data structure |
US7493654B2 (en) * | 2004-11-20 | 2009-02-17 | International Business Machines Corporation | Virtualized protective communications system |
US7496036B2 (en) * | 2004-11-22 | 2009-02-24 | International Business Machines Corporation | Method and apparatus for determining client-perceived server response time |
US7974216B2 (en) | 2004-11-22 | 2011-07-05 | Cisco Technology, Inc. | Approach for determining the real time availability of a group of network elements |
US7529196B2 (en) * | 2004-12-07 | 2009-05-05 | Hewlett-Packard Development Company, L.P. | Routing a service query in an overlay network |
US7548562B2 (en) * | 2004-12-14 | 2009-06-16 | Agilent Technologies, Inc. | High speed acquisition system that allows capture from a packet network and streams the data to a storage medium |
EP1832054B1 (en) | 2004-12-23 | 2018-03-21 | Symantec Corporation | Method and apparatus for network packet capture distributed storage system |
US20060165009A1 (en) | 2005-01-25 | 2006-07-27 | Zvolve | Systems and methods for traffic management between autonomous systems in the Internet |
US7453804B1 (en) | 2005-02-08 | 2008-11-18 | Packeteer, Inc. | Aggregate network resource utilization control scheme |
US7418006B2 (en) | 2005-03-08 | 2008-08-26 | Microsoft Corporation | Virtual endpoints |
US7420992B1 (en) | 2005-03-17 | 2008-09-02 | Packeteer, Inc. | Adaptive network traffic compression mechanism including dynamic selection of compression algorithms |
US7480238B2 (en) * | 2005-04-14 | 2009-01-20 | International Business Machines Corporation | Dynamic packet training |
US7561569B2 (en) * | 2005-07-11 | 2009-07-14 | Battelle Memorial Institute | Packet flow monitoring tool and method |
US7522521B2 (en) * | 2005-07-12 | 2009-04-21 | Cisco Technology, Inc. | Route processor adjusting of line card admission control parameters for packets destined for the route processor |
US7907608B2 (en) | 2005-08-12 | 2011-03-15 | Mcafee, Inc. | High speed packet capture |
US8077718B2 (en) * | 2005-08-12 | 2011-12-13 | Microsoft Corporation | Distributed network management |
US7532633B2 (en) * | 2005-10-12 | 2009-05-12 | Juniper Networks, Inc. | Spoof checking within a label switching computer network |
JP4648181B2 (en) | 2005-12-16 | 2011-03-09 | 富士通株式会社 | Data analysis apparatus, data analysis method, and program thereof |
TWI301025B (en) | 2005-12-28 | 2008-09-11 | Ind Tech Res Inst | Method for transmitting real-time streaming data and apparatus using the same |
US7797740B2 (en) | 2006-01-06 | 2010-09-14 | Nokia Corporation | System and method for managing captured content |
JP4673752B2 (en) * | 2006-01-13 | 2011-04-20 | 株式会社日立製作所 | Multicast packet controller |
US7466694B2 (en) | 2006-06-10 | 2008-12-16 | Cisco Technology, Inc. | Routing protocol with packet network attributes for improved route selection |
CN100461732C (en) | 2006-06-16 | 2009-02-11 | 华为技术有限公司 | Ethernet technology switching and forwarding method, system and equipment |
WO2008037114A1 (en) | 2006-09-25 | 2008-04-03 | Huawei Technologies Co., Ltd. | Information carrying synchronization code and method for frame timing synchronization |
US20080117903A1 (en) | 2006-10-20 | 2008-05-22 | Sezen Uysal | Apparatus and method for high speed and large amount of data packet capturing and replaying |
US7836169B2 (en) | 2007-01-24 | 2010-11-16 | Cisco Technology, Inc. | Method and system for identifying and reporting over-utilized, under-utilized, and bad quality trunks and gateways in internet protocol telephony networks |
EP1971087A1 (en) | 2007-03-12 | 2008-09-17 | Nokia Siemens Networks Gmbh & Co. Kg | Method for controlling data traffic between gateways |
US8185355B2 (en) | 2007-04-03 | 2012-05-22 | Microsoft Corporation | Slot-cache for caching aggregates of data with different expiry times |
US8756350B2 (en) | 2007-06-26 | 2014-06-17 | International Business Machines Corporation | Method and apparatus for efficiently tracking queue entries relative to a timestamp |
US8897211B2 (en) | 2007-06-29 | 2014-11-25 | Alcatel Lucent | System and methods for providing service-specific support for multimedia traffic in wireless networks |
US8988995B2 (en) | 2007-07-23 | 2015-03-24 | Mitel Network Corporation | Network traffic management |
US20090028169A1 (en) | 2007-07-27 | 2009-01-29 | Motorola, Inc. | Method and device for routing mesh network traffic |
US8130656B2 (en) | 2007-08-07 | 2012-03-06 | Motorola Solutions, Inc. | Method and device for routing mesh network traffic |
US8250641B2 (en) | 2007-09-17 | 2012-08-21 | Intel Corporation | Method and apparatus for dynamic switching and real time security control on virtualized systems |
US20090092057A1 (en) | 2007-10-09 | 2009-04-09 | Latis Networks, Inc. | Network Monitoring System with Enhanced Performance |
US20090097418A1 (en) | 2007-10-11 | 2009-04-16 | Alterpoint, Inc. | System and method for network service path analysis |
US8625610B2 (en) | 2007-10-12 | 2014-01-07 | Cisco Technology, Inc. | System and method for improving spoke to spoke communication in a computer network |
US8559319B2 (en) | 2007-10-19 | 2013-10-15 | Voxer Ip Llc | Method and system for real-time synchronization across a distributed services communication network |
IL187046A0 (en) | 2007-10-30 | 2008-02-09 | Sandisk Il Ltd | Memory randomization for protection against side channel attacks |
DE102007052180A1 (en) | 2007-10-31 | 2009-05-07 | Fujitsu Siemens Computers Gmbh | Method, computer system and computer program product |
US9106450B2 (en) | 2007-11-01 | 2015-08-11 | International Business Machines Corporation | System and method for communication management |
US7529932B1 (en) * | 2008-03-31 | 2009-05-05 | International Business Machines Corporation | Removable medium and system and method for writing data to same |
-
2005
- 2005-12-16 EP EP05854314.1A patent/EP1832054B1/en not_active Not-in-force
- 2005-12-16 WO PCT/US2005/045566 patent/WO2006071560A2/en active Application Filing
- 2005-12-16 JP JP2007548320A patent/JP4759574B2/en not_active Expired - Fee Related
- 2005-12-16 CA CA2619141A patent/CA2619141C/en active Active
- 2005-12-16 US US11/632,249 patent/US7855974B2/en not_active Expired - Fee Related
- 2005-12-16 AU AU2005322350A patent/AU2005322350B2/en not_active Ceased
-
2009
- 2009-04-01 US US12/416,276 patent/US20090182953A1/en not_active Abandoned
- 2009-05-21 US US12/469,744 patent/US7684347B2/en not_active Expired - Fee Related
-
2011
- 2011-03-28 JP JP2011070214A patent/JP2011222006A/en active Pending
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS59127453A (en) | 1983-01-10 | 1984-07-23 | Nec Corp | Line monitoring device |
US20030135525A1 (en) | 2001-07-17 | 2003-07-17 | Huntington Stephen Glen | Sliding window packet management systems |
Non-Patent Citations (1)
Title |
---|
See also references of EP1832054A4 |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009021573A1 (en) * | 2007-08-14 | 2009-02-19 | Rohde & Schwarz Gmbh & Co. Kg | Method and device for logging communications connections at very high data rates |
US8631141B2 (en) | 2007-08-14 | 2014-01-14 | Rohde & Schwarz Gmbh & Co. Kg | Method and device for logging communications connections at very high data rates |
WO2011060377A1 (en) | 2009-11-15 | 2011-05-19 | Solera Networks, Inc. | Method and apparatus for real time identification and recording of artifacts |
WO2011060368A1 (en) | 2009-11-15 | 2011-05-19 | Solera Networks, Inc. | Method and apparatus for storing and indexing high-speed network traffic data |
US9349024B2 (en) | 2011-01-18 | 2016-05-24 | International Business Machines Corporation | Assigning a data item to a storage location in a computing environment |
US9948714B2 (en) | 2011-01-18 | 2018-04-17 | International Business Machines Corporation | Assigning a data item to a storage location in a computing environment |
WO2013006185A1 (en) | 2011-07-06 | 2013-01-10 | Gigamon Llc | Network switch with traffic generation capability |
EP2729879B1 (en) * | 2011-07-06 | 2019-07-17 | Gigamon Inc. | Network switch with traffic generation capability |
TWI711285B (en) * | 2019-09-18 | 2020-11-21 | 緯創資通股份有限公司 | Network failure detection method and network failure detection device |
Also Published As
Publication number | Publication date |
---|---|
US20090182953A1 (en) | 2009-07-16 |
CA2619141A1 (en) | 2006-07-06 |
US20070248029A1 (en) | 2007-10-25 |
EP1832054A2 (en) | 2007-09-12 |
AU2005322350A2 (en) | 2006-07-06 |
AU2005322350B2 (en) | 2010-10-21 |
WO2006071560A3 (en) | 2006-08-17 |
AU2005322350A1 (en) | 2006-07-06 |
JP4759574B2 (en) | 2011-08-31 |
US7684347B2 (en) | 2010-03-23 |
EP1832054A4 (en) | 2016-08-17 |
JP2008526109A (en) | 2008-07-17 |
EP1832054B1 (en) | 2018-03-21 |
CA2619141C (en) | 2014-10-21 |
US7855974B2 (en) | 2010-12-21 |
US20090219829A1 (en) | 2009-09-03 |
JP2011222006A (en) | 2011-11-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CA2619141C (en) | Method and apparatus for network packet capture distributed storage system | |
US20100195538A1 (en) | Method and apparatus for network packet capture distributed storage system | |
US8112395B2 (en) | Systems and methods for providing a distributed file system utilizing metadata to track information about data stored throughout the system | |
JP5096441B2 (en) | Method for file restriping in a distributed file system | |
US7509524B2 (en) | Systems and methods for a distributed file system with data recovery | |
US7272613B2 (en) | Method and system for managing distributed content and related metadata | |
EP1892921B1 (en) | Method and system for managing distributed content and related metadata | |
US7149189B2 (en) | Network data retrieval and filter systems and methods | |
US7406473B1 (en) | Distributed file system using disk servers, lock servers and file servers | |
US9571356B2 (en) | Capturing data packets from external networks into high availability clusters while maintaining high availability of popular data packets | |
US10756952B2 (en) | Determining a storage network path utilizing log data | |
US11218391B2 (en) | Methods for monitoring performance of a network fabric and devices thereof | |
US20160294948A1 (en) | System for database, application, and storage security in software defined network | |
Moore et al. | Configuring and tuning archival storage systems | |
Olker et al. | Optimizing NFS Performance: Tuning and Troubleshooting NFS on HP-UX Systems | |
Lu | Design issues in networked intelligent storage systems: Performance and QoS |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
WWE | Wipo information: entry into national phase |
Ref document number: 11632249 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2005322350 Country of ref document: AU |
|
WWE | Wipo information: entry into national phase |
Ref document number: 555615 Country of ref document: NZ |
|
ENP | Entry into the national phase |
Ref document number: 2005322350 Country of ref document: AU Date of ref document: 20051216 Kind code of ref document: A |
|
WWP | Wipo information: published in national office |
Ref document number: 2005322350 Country of ref document: AU |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2007548320 Country of ref document: JP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2005854314 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1098/MUMNP/2007 Country of ref document: IN |
|
WWP | Wipo information: published in national office |
Ref document number: 2005854314 Country of ref document: EP |
|
WWP | Wipo information: published in national office |
Ref document number: 11632249 Country of ref document: US |
|
ENP | Entry into the national phase |
Ref document number: 2619141 Country of ref document: CA |