WITS: Waikato VI

Trace Format ERF, captured using a DAG 3 card.
Volume on Disk 495 GB
Number of Traces 155
Capture Start (Local) Wed Oct 31 16:06:26 2007
Capture End (Local) Sat Mar 15 03:21:28 2008
Total Duration 135 Days, 11 Hours, 15 Minutes and 2 Seconds
Packets Captured 20,886 million
Total Traffic 11,092 GB
Contiguity No gaps whatsoever
Snapping Method Packets truncated four bytes after the end of the transport header, except for DNS
Rotation Policy Daily rotation at Midnight UTC. Also rotate on AES key change.
Anonymization IP addresses anonymized using Crypto-Pan AES encryption.

This is a contiguous packet header trace captured at the border of the University of Waikato network. The traces were captured using a single DAG 3 card and the WDCap trace capture software. The version of WDCap used was version 3.0.6 and the Libtrace version was 3.0.1.

The capture point was located between the University's network infrastructure and the commodity Internet. This allowed access to all the traffic that was coming into and exiting the University. However, no internal traffic would have been observed and captured by our capture point. The capture machine performed all the anonymization and truncation before exporting the packets via the network to a second machine. That machine was also running WDCap which would read the packets off the network and write the traces.

Each trace file is named using the following format: yyyymmdd-HHMMSS-[code].gz. The time and date refers to the time in UTC when the first packet in the file was captured. The code refers to the event which caused the previous file to be closed and this new file to be created. Note that new codes have been added in this edition of WDCap.

Codes of interest for this traceset are as follows:

  • 0 - Rotation boundary was reached
  • 1 - Encryption key was changed
  • 4 - The capture process has been restarted

Regular file rotation (code 0) occured daily at Midnight (UTC).

Packet records are truncated four bytes after the end of the transport header except in the case of DNS traffic, which is snapped twelve bytes after the end of the transport header. This means that many packets will contain a small amount of user payload - enough to perform some rudimentary layer 7 analysis without seriously threatening the privacy of the network users. ICMP packets which are truncated after any IP and transport headers that may be present in the packet payload.

The IP addresses contained within the packets have been anonymised using Crypto-Pan AES encryption, which is a prefix-preserving anonymisation method. This means that unanonymised IP addresses that were on the same subnet will also be identifiable as on the same subnet when the addresses are anonymized. We change the encryption key once a week on Sunday midnight (local time). This key change causes a file rotation, with a rotation code of 1.

The TCP and IP checksums have also been validated and anonymized. If the checksum was correct, it has been replaced with 0. If the checksum was incorrect, it has been replaced with 1.

The recommended method for processing these traces is to use Libtrace, which we have developed. There are a number of tools included with libtrace such as a packet dumping utility, a trace format converter (for example, to convert to pcap), a trace splitting/filtering tool and a few statistic generators. We suggest you examine the Libtrace Wiki for more details on the Libtrace tools and the library itself.

Name Local Start Time Duration Total Packets Compressed Size
20071031-030626-4 Wed Oct 31 16:06:26 2007 20:53:35 206 million 5,206 MB
20071101-000000-0 Thu Nov 1 13:00:01 2007 24:00:00 233 million 6,025 MB
20071102-000000-0 Fri Nov 2 13:00:01 2007 24:00:00 237 million 6,207 MB
20071103-000000-0 Sat Nov 3 13:00:01 2007 11:00:01 115 million 3,040 MB
20071103-110002-1 Sun Nov 4 00:00:02 2007 12:59:59 94 million 2,379 MB
20071104-000000-0 Sun Nov 4 13:00:01 2007 24:00:00 251 million 6,486 MB
20071105-000000-0 Mon Nov 5 13:00:01 2007 24:00:00 267 million 6,967 MB
20071106-000000-0 Tue Nov 6 13:00:01 2007 24:00:00 263 million 6,804 MB
20071107-000000-0 Wed Nov 7 13:00:01 2007 24:00:00 285 million 7,405 MB
20071108-000000-0 Thu Nov 8 13:00:01 2007 24:00:00 264 million 6,709 MB
20071109-000000-0 Fri Nov 9 13:00:01 2007 24:00:00 199 million 5,056 MB
20071110-000000-0 Sat Nov 10 13:00:01 2007 11:00:01 53 million 1,302 MB
20071110-110002-1 Sun Nov 11 00:00:02 2007 12:59:59 55 million 1,337 MB
20071111-000000-0 Sun Nov 11 13:00:01 2007 24:00:00 134 million 3,258 MB
20071112-000000-0 Mon Nov 12 13:00:01 2007 24:00:00 160 million 3,903 MB
20071113-000000-0 Tue Nov 13 13:00:01 2007 24:00:00 134 million 3,227 MB
20071114-000000-0 Wed Nov 14 13:00:01 2007 24:00:00 147 million 3,528 MB
20071115-000000-0 Thu Nov 15 13:00:01 2007 24:00:00 177 million 4,291 MB
20071116-000000-0 Fri Nov 16 13:00:01 2007 24:00:00 109 million 2,615 MB
20071117-000000-0 Sat Nov 17 13:00:01 2007 11:00:01 33 million 802 MB
20071117-110002-1 Sun Nov 18 00:00:02 2007 12:59:59 29 million 687 MB
20071118-000000-0 Sun Nov 18 13:00:01 2007 24:00:00 118 million 2,824 MB
20071119-000000-0 Mon Nov 19 13:00:01 2007 24:00:00 141 million 3,386 MB
20071120-000000-0 Tue Nov 20 13:00:01 2007 24:00:00 147 million 3,632 MB
20071121-000000-0 Wed Nov 21 13:00:01 2007 24:00:00 177 million 4,456 MB
20071122-000000-0 Thu Nov 22 13:00:01 2007 24:00:00 173 million 4,297 MB
20071123-000000-0 Fri Nov 23 13:00:01 2007 24:00:00 123 million 2,977 MB
20071124-000000-0 Sat Nov 24 13:00:01 2007 11:00:01 56 million 1,373 MB
20071124-110002-1 Sun Nov 25 00:00:02 2007 12:59:59 43 million 1,030 MB
20071125-000000-0 Sun Nov 25 13:00:01 2007 24:00:00 106 million 2,549 MB
20071126-000000-0 Mon Nov 26 13:00:01 2007 24:00:00 153 million 3,691 MB
20071127-000000-0 Tue Nov 27 13:00:01 2007 24:00:00 143 million 3,371 MB
20071128-000000-0 Wed Nov 28 13:00:01 2007 24:00:00 147 million 3,524 MB
20071129-000000-0 Thu Nov 29 13:00:01 2007 24:00:00 171 million 4,004 MB
20071130-000000-0 Fri Nov 30 13:00:01 2007 24:00:00 120 million 2,837 MB
20071201-000000-0 Sat Dec 1 13:00:01 2007 11:00:01 31 million 711 MB
20071201-110002-1 Sun Dec 2 00:00:02 2007 12:59:59 30 million 720 MB
20071202-000000-0 Sun Dec 2 13:00:01 2007 24:00:00 178 million 4,725 MB
20071203-000000-0 Mon Dec 3 13:00:01 2007 24:00:00 152 million 3,635 MB
20071204-000000-0 Tue Dec 4 13:00:01 2007 24:00:00 153 million 3,658 MB
20071205-000000-0 Wed Dec 5 13:00:01 2007 24:00:00 131 million 3,144 MB
20071206-000000-0 Thu Dec 6 13:00:01 2007 24:00:00 122 million 2,937 MB
20071207-000000-0 Fri Dec 7 13:00:01 2007 24:00:00 111 million 2,641 MB
20071208-000000-0 Sat Dec 8 13:00:01 2007 11:00:01 31 million 750 MB
20071208-110002-1 Sun Dec 9 00:00:02 2007 12:59:59 40 million 948 MB
20071209-000000-0 Sun Dec 9 13:00:01 2007 24:00:00 110 million 2,635 MB
20071210-000000-0 Mon Dec 10 13:00:01 2007 24:00:00 156 million 3,756 MB
20071211-000000-0 Tue Dec 11 13:00:01 2007 24:00:00 187 million 4,584 MB
20071212-000000-0 Wed Dec 12 13:00:01 2007 24:00:00 146 million 3,513 MB
20071213-000000-0 Thu Dec 13 13:00:01 2007 24:00:00 146 million 3,526 MB
20071214-000000-0 Fri Dec 14 13:00:01 2007 24:00:00 102 million 2,480 MB
20071215-000000-0 Sat Dec 15 13:00:01 2007 11:00:01 35 million 850 MB
20071215-110002-1 Sun Dec 16 00:00:02 2007 12:59:59 35 million 850 MB
20071216-000000-0 Sun Dec 16 13:00:01 2007 24:00:00 130 million 3,222 MB
20071217-000000-0 Mon Dec 17 13:00:01 2007 24:00:00 171 million 4,236 MB
20071218-000000-0 Tue Dec 18 13:00:01 2007 24:00:00 162 million 3,817 MB
20071219-000000-0 Wed Dec 19 13:00:01 2007 24:00:00 154 million 3,719 MB
20071220-000000-0 Thu Dec 20 13:00:01 2007 24:00:00 140 million 3,380 MB
20071221-000000-0 Fri Dec 21 13:00:01 2007 24:00:00 88 million 2,089 MB
20071222-000000-0 Sat Dec 22 13:00:01 2007 11:00:01 43 million 1,003 MB
20071222-110001-1 Sun Dec 23 00:00:01 2007 12:59:59 36 million 864 MB
20071223-000000-0 Sun Dec 23 13:00:01 2007 24:00:00 89 million 2,028 MB
20071224-000000-0 Mon Dec 24 13:00:01 2007 24:00:00 81 million 1,855 MB
20071225-000000-0 Tue Dec 25 13:00:01 2007 24:00:00 73 million 1,689 MB
20071226-000000-0 Wed Dec 26 13:00:01 2007 24:00:00 83 million 1,925 MB
20071227-000000-0 Thu Dec 27 13:00:01 2007 24:00:00 82 million 1,955 MB
20071228-000000-0 Fri Dec 28 13:00:01 2007 24:00:00 87 million 2,074 MB
20071229-000000-0 Sat Dec 29 13:00:01 2007 11:00:02 34 million 800 MB
20071229-110002-1 Sun Dec 30 00:00:02 2007 12:59:58 33 million 799 MB
20071230-000000-0 Sun Dec 30 13:00:01 2007 24:00:00 72 million 1,689 MB
20071231-000000-0 Mon Dec 31 13:00:01 2007 24:00:00 60 million 1,406 MB
20080101-000000-0 Tue Jan 1 13:00:01 2008 24:00:00 68 million 1,554 MB
20080102-000000-0 Wed Jan 2 13:00:01 2008 24:00:00 86 million 1,990 MB
20080103-000000-0 Thu Jan 3 13:00:01 2008 24:00:00 114 million 2,707 MB
20080104-000000-0 Fri Jan 4 13:00:01 2008 24:00:00 94 million 2,233 MB
20080105-000000-0 Sat Jan 5 13:00:01 2008 11:00:02 31 million 730 MB
20080105-110002-1 Sun Jan 6 00:00:02 2008 12:59:58 35 million 827 MB
20080106-000000-0 Sun Jan 6 13:00:01 2008 24:00:00 104 million 2,487 MB
20080107-000000-0 Mon Jan 7 13:00:01 2008 24:00:00 215 million 5,108 MB
20080108-000000-0 Tue Jan 8 13:00:01 2008 24:00:00 172 million 4,047 MB
20080109-000000-0 Wed Jan 9 13:00:01 2008 24:00:00 207 million 4,539 MB
20080110-000000-0 Thu Jan 10 13:00:01 2008 24:00:00 219 million 4,997 MB
20080111-000000-0 Fri Jan 11 13:00:01 2008 24:00:00 141 million 3,482 MB
20080112-000000-0 Sat Jan 12 13:00:01 2008 11:00:01 63 million 1,529 MB
20080112-110002-1 Sun Jan 13 00:00:02 2008 12:59:59 59 million 1,469 MB
20080113-000000-0 Sun Jan 13 13:00:01 2008 24:00:00 145 million 3,609 MB
20080114-000000-0 Mon Jan 14 13:00:01 2008 24:00:00 193 million 4,635 MB
20080115-000000-0 Tue Jan 15 13:00:01 2008 24:00:00 170 million 4,044 MB
20080116-000000-0 Wed Jan 16 13:00:01 2008 24:00:00 171 million 4,046 MB
20080117-000000-0 Thu Jan 17 13:00:01 2008 24:00:00 148 million 3,525 MB
20080118-000000-0 Fri Jan 18 13:00:01 2008 24:00:00 121 million 2,910 MB
20080119-000000-0 Sat Jan 19 13:00:01 2008 11:00:02 45 million 1,088 MB
20080119-110002-1 Sun Jan 20 00:00:02 2008 12:59:58 43 million 1,013 MB
20080120-000000-0 Sun Jan 20 13:00:01 2008 24:00:00 128 million 3,038 MB
20080121-000000-0 Mon Jan 21 13:00:01 2008 24:00:00 166 million 3,971 MB
20080122-000000-0 Tue Jan 22 13:00:01 2008 24:00:00 218 million 5,262 MB
20080123-000000-0 Wed Jan 23 13:00:01 2008 24:00:00 161 million 3,914 MB
20080124-000000-0 Thu Jan 24 13:00:01 2008 24:00:00 167 million 4,020 MB
20080125-000000-0 Fri Jan 25 13:00:01 2008 24:00:00 128 million 3,080 MB
20080126-000000-0 Sat Jan 26 13:00:01 2008 11:00:02 44 million 1,062 MB
20080126-110002-1 Sun Jan 27 00:00:02 2008 12:59:58 46 million 1,094 MB
20080127-000000-0 Sun Jan 27 13:00:01 2008 24:00:00 108 million 2,566 MB
20080128-000000-0 Mon Jan 28 13:00:01 2008 24:00:00 145 million 3,443 MB
20080129-000000-0 Tue Jan 29 13:00:01 2008 24:00:00 181 million 4,500 MB
20080130-000000-0 Wed Jan 30 13:00:01 2008 24:00:00 168 million 4,125 MB
20080131-000000-0 Thu Jan 31 13:00:01 2008 24:00:00 155 million 3,770 MB
20080201-000000-0 Fri Feb 1 13:00:01 2008 24:00:00 115 million 2,732 MB
20080202-000000-0 Sat Feb 2 13:00:01 2008 11:00:02 41 million 963 MB
20080202-110002-1 Sun Feb 3 00:00:02 2008 12:59:58 38 million 897 MB
20080203-000000-0 Sun Feb 3 13:00:01 2008 24:00:00 123 million 2,912 MB
20080204-000000-0 Mon Feb 4 13:00:01 2008 24:00:00 151 million 3,636 MB
20080205-000000-0 Tue Feb 5 13:00:01 2008 24:00:00 140 million 3,369 MB
20080206-000000-0 Wed Feb 6 13:00:01 2008 24:00:00 131 million 3,184 MB
20080207-000000-0 Thu Feb 7 13:00:01 2008 24:00:00 198 million 4,950 MB
20080208-000000-0 Fri Feb 8 13:00:01 2008 24:00:00 126 million 2,999 MB
20080209-000000-0 Sat Feb 9 13:00:01 2008 11:00:01 38 million 902 MB
20080209-110002-1 Sun Feb 10 00:00:02 2008 12:59:59 35 million 835 MB
20080210-000000-0 Sun Feb 10 13:00:01 2008 24:00:00 152 million 3,758 MB
20080211-000000-0 Mon Feb 11 13:00:01 2008 24:00:00 168 million 4,105 MB
20080212-000000-0 Tue Feb 12 13:00:01 2008 24:00:00 177 million 4,331 MB
20080213-000000-0 Wed Feb 13 13:00:01 2008 24:00:00 214 million 5,409 MB
20080214-000000-0 Thu Feb 14 13:00:01 2008 24:00:00 229 million 5,841 MB
20080215-000000-0 Fri Feb 15 13:00:01 2008 24:00:00 134 million 3,298 MB
20080216-000000-0 Sat Feb 16 13:00:01 2008 11:00:01 52 million 1,257 MB
20080216-110001-1 Sun Feb 17 00:00:01 2008 12:59:59 48 million 1,150 MB
20080217-000000-0 Sun Feb 17 13:00:01 2008 24:00:00 134 million 3,229 MB
20080218-000000-0 Mon Feb 18 13:00:01 2008 24:00:00 184 million 4,439 MB
20080219-000000-0 Tue Feb 19 13:00:01 2008 24:00:00 189 million 4,561 MB
20080220-000000-0 Wed Feb 20 13:00:01 2008 24:00:00 220 million 5,549 MB
20080221-000000-0 Thu Feb 21 13:00:01 2008 24:00:00 241 million 5,991 MB
20080222-000000-0 Fri Feb 22 13:00:01 2008 24:00:00 168 million 4,100 MB
20080223-000000-0 Sat Feb 23 13:00:01 2008 11:00:01 68 million 1,671 MB
20080223-110002-1 Sun Feb 24 00:00:02 2008 12:59:59 69 million 1,696 MB
20080224-000000-0 Sun Feb 24 13:00:01 2008 24:00:00 184 million 4,544 MB
20080225-000000-0 Mon Feb 25 13:00:01 2008 24:00:00 205 million 5,057 MB
20080226-000000-0 Tue Feb 26 13:00:01 2008 24:00:00 196 million 4,818 MB
20080227-000000-0 Wed Feb 27 13:00:01 2008 24:00:00 190 million 4,641 MB
20080228-000000-0 Thu Feb 28 13:00:01 2008 24:00:00 207 million 5,146 MB
20080229-000000-0 Fri Feb 29 13:00:01 2008 24:00:00 172 million 4,103 MB
20080301-000000-0 Sat Mar 1 13:00:01 2008 11:00:01 73 million 1,744 MB
20080301-110002-1 Sun Mar 2 00:00:02 2008 12:59:59 62 million 1,464 MB
20080302-000000-0 Sun Mar 2 13:00:01 2008 24:00:00 190 million 4,500 MB
20080303-000000-0 Mon Mar 3 13:00:01 2008 24:00:00 232 million 5,603 MB
20080304-000000-0 Tue Mar 4 13:00:01 2008 24:00:00 243 million 5,989 MB
20080305-000000-0 Wed Mar 5 13:00:01 2008 24:00:00 242 million 6,014 MB
20080306-000000-0 Thu Mar 6 13:00:01 2008 24:00:00 236 million 5,816 MB
20080307-000000-0 Fri Mar 7 13:00:01 2008 24:00:00 164 million 3,981 MB
20080308-000000-0 Sat Mar 8 13:00:01 2008 11:00:02 58 million 1,416 MB
20080308-110002-1 Sun Mar 9 00:00:02 2008 12:59:58 53 million 1,318 MB
20080309-000000-0 Sun Mar 9 13:00:01 2008 24:00:00 166 million 4,078 MB
20080310-000000-0 Mon Mar 10 13:00:01 2008 24:00:00 312 million 7,636 MB
20080311-000000-0 Tue Mar 11 13:00:01 2008 24:00:00 225 million 5,576 MB
20080312-000000-0 Wed Mar 12 13:00:01 2008 24:00:00 229 million 5,635 MB
20080313-000000-0 Thu Mar 13 13:00:01 2008 24:00:00 223 million 5,550 MB
20080314-000000-0 Fri Mar 14 13:00:01 2008 14:21:27 128 million 3,192 MB