This is a mostly continuous packet header trace captured from a New Zealand ISP (which must remain unnamed). The traces were captured using a single DAG 3.7G card and the WDCap trace capture software. The version of WDCap used was version 3.1.0 and the Libtrace version was 3.0.6.

The capture point was connected to a mirrored port on a switch that carried traffic to and from the core routers for the ISP. This configuration enabled us to capture all the traffic for the ISP's customers (and some internal stuff as well) in both directions. This particular traceset has been created by filtering the original capture to only include traffic involving the ISP's residential DSL subscribers.

No network export was involved in the capture process - all packets were captured and written to disk using the same WDCap process.

Each trace file is named using the following format: yyyymmdd-HHMMSS-[code].gz. The time and date refers to the time in UTC when the first packet in the file was captured. The code refers to the event which caused the previous file to be closed and this new file to be created.

Codes of interest for this traceset are as follows:

• 0 - Rotation boundary was reached
• 2 - DAG card dropped packets

Regular file rotation (code 0) occurs twice every hour, starting at the beginning of the hour. There are two gaps in the trace set. The first gap (between 20090106-010000-0 and 20090106-024703-0) marks the end of a short set of traces that were saved when testing the capture setup. The capture proper began with 20090106-024703-0 and ran uninterrupted until 20090114-004658-2, when an unknown error caused the capture buffer on the DAG card to overflow. As a result, some packets that passed the capture point were not captured and recorded to disk. Shortly afterward, the WDCap process was able to catch up and resumed writing traces normally until the end of the trace set.

Packet records are truncated four bytes after the end of the transport header except in the case of DNS traffic, which is snapped twelve bytes after the end of the transport header. This means that many packets will contain a small amount of user payload - enough to perform some rudimentary layer 7 analysis without seriously threatening the privacy of the network users. ICMP packets which are truncated after any IP and transport headers that may be present in the packet payload.

The IP addresses contained within the packets have been anonymised using Crypto-Pan AES encryption, which is a prefix-preserving anonymisation method. This means that unanonymised IP addresses that were on the same subnet will also be identifiable as on the same subnet when the addresses are anonymized. The same encryption key has been used throughout the entire traceset, so IP anonymisation is consistent in all trace files.

The recommended method for processing these traces is to use Libtrace, which we have developed. There are a number of tools included with libtrace such as a packet dumping utility, a trace format converter (for example, to convert to pcap), a trace splitting/filtering tool and a few statistic generators. We suggest you examine the Libtrace Wiki for more details on the Libtrace tools and the library itself.