WITS: Waikato IV

Trace Format ERF, captured using a DAG 3 card.
Volume on Disk 255 GB
Number of Traces 65
Capture Start (Local) Wed Mar 28 16:53:13 2007
Capture End (Local) Wed May 23 21:35:38 2007
Total Duration 56 Days, 4 Hours, 42 Minutes and 24 Seconds
Packets Captured 10,128 million
Total Traffic 4,588 GB
Contiguity No gaps whatsoever.
Snapping Method Packets truncated four bytes after the end of the transport header, except for DNS
Rotation Policy Daily rotation at Midnight UTC. Also rotate on AES key change.
Anonymization IP addresses anonymized using Crypto-Pan AES encryption.

This is a contiguous packet header trace captured at the border of the University of Waikato network. The traces were captured using a single DAG 3 card and the WDCap trace capture software. The version of WDCap used was version 3.0.6 and the Libtrace version was 3.0.1.

The capture point was located between the University's network infrastructure and the commodity Internet. This allowed access to all the traffic that was coming into and exiting the University. However, no internal traffic would have been observed and captured by our capture point. The capture machine performed all the anonymization and truncation before exporting the packets via the network to a second machine. That machine was also running WDCap which would read the packets off the network and write the traces.

Each trace file is named using the following format: yyyymmdd-HHMMSS-[code].gz. The time and date refers to the time in UTC when the first packet in the file was captured. The code refers to the event which caused the previous file to be closed and this new file to be created. Note that new codes have been added in this edition of WDCap.

Codes of interest for this traceset are as follows:

  • 0 - Rotation boundary was reached
  • 1 - Encryption key was changed
  • 4 - The capture process has been restarted

Regular file rotation (code 0) occured daily at Midnight (UTC).

Packet records are truncated four bytes after the end of the transport header except in the case of DNS traffic, which is snapped twelve bytes after the end of the transport header. This means that many packets will contain a small amount of user payload - enough to perform some rudimentary layer 7 analysis without seriously threatening the privacy of the network users. ICMP packets which are truncated after any IP and transport headers that may be present in the packet payload.

The IP addresses contained within the packets have been anonymised using Crypto-Pan AES encryption, which is a prefix-preserving anonymisation method. This means that unanonymised IP addresses that were on the same subnet will also be identifiable as on the same subnet when the addresses are anonymized. We change the encryption key once a week on Sunday midnight (local time). This key change causes a file rotation, with a rotation code of 1.

The TCP and IP checksums have also been validated and anonymized. If the checksum was correct, it has been replaced with 0. If the checksum was incorrect, it has been replaced with 1.

The recommended method for processing these traces is to use Libtrace, which we have developed. There are a number of tools included with libtrace such as a packet dumping utility, a trace format converter (for example, to convert to pcap), a trace splitting/filtering tool and a few statistic generators. We suggest you examine the Libtrace Wiki for more details on the Libtrace tools and the library itself.

Name Local Start Time Duration Total Packets Compressed Size
20070328-045313-4 Wed Mar 28 16:53:13 2007 19:06:47 138 million 3,565 MB
20070329-000000-0 Thu Mar 29 12:00:01 2007 24:00:00 198 million 5,118 MB
20070330-000000-0 Fri Mar 30 12:00:01 2007 24:00:00 151 million 3,876 MB
20070331-000000-0 Sat Mar 31 12:00:01 2007 12:00:01 69 million 1,783 MB
20070331-120001-1 Sun Apr 1 00:00:01 2007 11:59:59 43 million 1,136 MB
20070401-000000-0 Sun Apr 1 12:00:01 2007 24:00:00 157 million 4,023 MB
20070402-000000-0 Mon Apr 2 12:00:01 2007 24:00:00 222 million 5,726 MB
20070403-000000-0 Tue Apr 3 12:00:01 2007 24:00:00 223 million 5,849 MB
20070404-000000-0 Wed Apr 4 12:00:01 2007 24:00:00 217 million 5,754 MB
20070405-000000-0 Thu Apr 5 12:00:01 2007 24:00:00 158 million 4,167 MB
20070406-000000-0 Fri Apr 6 12:00:01 2007 24:00:00 106 million 2,829 MB
20070407-000000-0 Sat Apr 7 12:00:01 2007 12:00:01 54 million 1,448 MB
20070407-120002-1 Sun Apr 8 00:00:02 2007 11:59:59 42 million 1,135 MB
20070408-000000-0 Sun Apr 8 12:00:01 2007 24:00:00 95 million 2,492 MB
20070409-000000-0 Mon Apr 9 12:00:01 2007 24:00:00 119 million 3,110 MB
20070410-000000-0 Tue Apr 10 12:00:01 2007 24:00:00 150 million 3,881 MB
20070411-000000-0 Wed Apr 11 12:00:01 2007 24:00:00 194 million 5,104 MB
20070412-000000-0 Thu Apr 12 12:00:01 2007 24:00:00 167 million 4,328 MB
20070413-000000-0 Fri Apr 13 12:00:01 2007 24:00:00 148 million 3,854 MB
20070414-000000-0 Sat Apr 14 12:00:01 2007 12:00:01 62 million 1,610 MB
20070414-120002-1 Sun Apr 15 00:00:02 2007 11:59:59 39 million 1,019 MB
20070415-000000-0 Sun Apr 15 12:00:01 2007 24:00:00 112 million 2,828 MB
20070416-000000-0 Mon Apr 16 12:00:01 2007 24:00:00 178 million 4,529 MB
20070417-000000-0 Tue Apr 17 12:00:01 2007 24:00:00 196 million 5,164 MB
20070418-000000-0 Wed Apr 18 12:00:01 2007 24:00:00 214 million 5,620 MB
20070419-000000-0 Thu Apr 19 12:00:01 2007 24:00:00 207 million 5,519 MB
20070420-000000-0 Fri Apr 20 12:00:01 2007 24:00:00 187 million 4,987 MB
20070421-000000-0 Sat Apr 21 12:00:01 2007 12:00:01 87 million 2,324 MB
20070421-120001-1 Sun Apr 22 00:00:01 2007 11:59:59 63 million 1,731 MB
20070422-000000-0 Sun Apr 22 12:00:01 2007 24:00:00 193 million 5,107 MB
20070423-000000-0 Mon Apr 23 12:00:01 2007 24:00:00 240 million 6,380 MB
20070424-000000-0 Tue Apr 24 12:00:01 2007 24:00:00 201 million 5,306 MB
20070425-000000-0 Wed Apr 25 12:00:01 2007 24:00:00 185 million 4,892 MB
20070426-000000-0 Thu Apr 26 12:00:01 2007 24:00:00 221 million 5,853 MB
20070427-000000-0 Fri Apr 27 12:00:01 2007 24:00:00 168 million 4,464 MB
20070428-000000-0 Sat Apr 28 12:00:01 2007 12:00:02 84 million 2,235 MB
20070428-120002-1 Sun Apr 29 00:00:02 2007 11:59:58 58 million 1,577 MB
20070429-000000-0 Sun Apr 29 12:00:01 2007 24:00:00 201 million 5,328 MB
20070430-000000-0 Mon Apr 30 12:00:01 2007 24:00:00 238 million 6,210 MB
20070501-000000-0 Tue May 1 12:00:01 2007 24:00:00 221 million 5,722 MB
20070502-000000-0 Wed May 2 12:00:01 2007 24:00:00 223 million 5,700 MB
20070503-000000-0 Thu May 3 12:00:01 2007 24:00:00 225 million 5,824 MB
20070504-000000-0 Fri May 4 12:00:01 2007 24:00:00 148 million 3,765 MB
20070505-000000-0 Sat May 5 12:00:01 2007 12:00:01 65 million 1,637 MB
20070505-120002-1 Sun May 6 00:00:02 2007 11:59:59 34 million 862 MB
20070506-000000-0 Sun May 6 12:00:01 2007 24:00:00 161 million 4,063 MB
20070507-000000-0 Mon May 7 12:00:01 2007 24:00:00 225 million 5,782 MB
20070508-000000-0 Tue May 8 12:00:01 2007 24:00:00 224 million 5,732 MB
20070509-000000-0 Wed May 9 12:00:01 2007 24:00:00 233 million 5,976 MB
20070510-000000-0 Thu May 10 12:00:01 2007 24:00:00 207 million 5,285 MB
20070511-000000-0 Fri May 11 12:00:01 2007 24:00:00 167 million 4,270 MB
20070512-000000-0 Sat May 12 12:00:01 2007 12:00:01 62 million 1,533 MB
20070512-120002-1 Sun May 13 00:00:02 2007 11:59:59 31 million 775 MB
20070513-000000-0 Sun May 13 12:00:01 2007 24:00:00 149 million 3,672 MB
20070514-000000-0 Mon May 14 12:00:01 2007 24:00:00 222 million 5,569 MB
20070515-000000-0 Tue May 15 12:00:01 2007 24:00:00 234 million 5,971 MB
20070516-000000-0 Wed May 16 12:00:01 2007 24:00:00 222 million 5,625 MB
20070517-000000-0 Thu May 17 12:00:01 2007 24:00:00 229 million 5,795 MB
20070518-000000-0 Fri May 18 12:00:01 2007 24:00:00 151 million 3,742 MB
20070519-000000-0 Sat May 19 12:00:01 2007 12:00:02 74 million 1,800 MB
20070519-120002-1 Sun May 20 00:00:02 2007 11:59:58 46 million 1,116 MB
20070520-000000-0 Sun May 20 12:00:01 2007 24:00:00 165 million 4,065 MB
20070521-000000-0 Mon May 21 12:00:01 2007 24:00:00 228 million 5,731 MB
20070522-000000-0 Tue May 22 12:00:01 2007 24:00:00 234 million 5,900 MB
20070523-000000-0 Wed May 23 12:00:01 2007 9:35:37 130 million 3,363 MB