WITS: Waikato III

Trace Format ERF, captured using DAG 3.
Volume on Disk 545 GB
Number of Traces 185
Capture Start (Local) Mon Sep 25 16:48:32 2006
Capture End (Local) Thu Mar 8 13:00:00 2007
Total Duration 160 Days, 11 Hours, 48 Minutes and 50 Seconds
Packets Captured 21,984 million
Total Traffic 9,144 GB
Contiguity Mostly contiguous, but there are a few gaps.
Snapping Method Packets truncated four bytes after the end of the transport header, except for DNS.
Rotation Policy Daily rotation at Midnight UTC. Also rotate on AES key change.
Anonymization IP addresses anonymized using Crypto-Pan AES encryption.

This is a continuous 5 1/2 month packet header trace captured at the border of the University of Waikato network. The traces were captured using a single DAG3 card and the WDCap trace capture software. The version of WDCap used was version 3.0.2 and the Libtrace version was 3.0.0.

The capture point was located between the University's network infrastructure and the commodity Internet. This allowed access to all the traffic that was coming into and exiting the University. However, no internal traffic would have been observed and captured by our capture point. The capture machine performed all the anonymization and truncation before exporting the packets via the network to a second machine. That machine was also running WDCap which would read the packets off the network and write the traces.

Each trace file is named using the following format: yyyymmdd-HHMMSS-[code].gz. The time and date refers to the time in UTC when the first packet in the file was captured. The code refers to the event which caused the previous file to be closed and this new file to be created.

Codes of interest for this traceset are as follows:

  • 0 - Rotation boundary was reached
  • 1 - Encryption key was changed

Regular file rotation (code 0) occured daily at Midnight (UTC). Note that there were a few software glitches when the capture of this traceset began, resulting in some small gaps in the capture. However, from October 2 through to February 7 the following year the traceset is completely contiguous without any gaps whatsoever.

Packet records are truncated four bytes after the end of the transport header except in the case of DNS traffic, which is snapped twelve bytes after the end of the transport header. This means that many packets will contain a small amount of user payload - enough to perform some rudimentary layer 7 analysis without seriously threatening the privacy of the network users. ICMP packets which are truncated after any IP and transport headers that may be present in the packet payload.

The IP addresses contained within the packets have been anonymised using Crypto-Pan AES encryption, which is a prefix-preserving anonymisation method. This means that unanonymised IP addresses that were on the same subnet will also be identifiable as on the same subnet when the addresses are anonymized. We change the encryption key once a week on Sunday midnight (local time). This key change causes a file rotation, with a rotation code of 1.

The TCP and IP checksums have also been validated and anonymized. If the checksum was correct, it has been replaced with 0. If the checksum was incorrect, it has been replaced with 1.

The recommended method for processing these traces is to use Libtrace, which we have developed. There are a number of tools included with libtrace such as a packet dumping utility, a trace format converter (for example, to convert to pcap), a trace splitting/filtering tool and a few statistic generators. We suggest you examine the Libtrace Wiki for more details on the Libtrace tools and the library itself.

Name Local Start Time Duration Total Packets Compressed Size
20060925-044832-0 Mon Sep 25 16:48:32 2006 0:01:22 <1 million 5 MB
20060925-045030-0 Mon Sep 25 16:50:30 2006 19:09:31 115 million 2,925 MB
20060926-000000-0 Tue Sep 26 12:00:01 2006 24:00:00 176 million 4,521 MB
20060927-000000-0 Wed Sep 27 12:00:01 2006 24:00:00 173 million 4,420 MB
20060928-000000-0 Thu Sep 28 12:00:01 2006 24:00:00 161 million 4,107 MB
20060929-000000-0 Fri Sep 29 12:00:01 2006 24:00:00 134 million 3,367 MB
20060930-000000-0 Sat Sep 30 12:00:01 2006 24:00:00 97 million 2,457 MB
20061001-000000-0 Sun Oct 1 13:00:01 2006 22:37:23 122 million 3,059 MB
20061001-224744-0 Mon Oct 2 11:47:44 2006 1:12:16 16 million 401 MB
20061002-000000-0 Mon Oct 2 13:00:01 2006 24:00:00 178 million 4,504 MB
20061003-000000-0 Tue Oct 3 13:00:01 2006 24:00:00 188 million 4,804 MB
20061004-000000-0 Wed Oct 4 13:00:01 2006 24:00:00 182 million 4,636 MB
20061005-000000-0 Thu Oct 5 13:00:01 2006 24:00:00 166 million 4,183 MB
20061006-000000-0 Fri Oct 6 13:00:01 2006 24:00:00 124 million 3,155 MB
20061007-000000-0 Sat Oct 7 13:00:01 2006 11:00:03 55 million 1,392 MB
20061007-110003-1 Sun Oct 8 00:00:03 2006 12:59:57 41 million 1,060 MB
20061008-000000-0 Sun Oct 8 13:00:01 2006 24:00:00 140 million 3,531 MB
20061009-000000-0 Mon Oct 9 13:00:01 2006 24:00:00 185 million 4,705 MB
20061010-000000-0 Tue Oct 10 13:00:01 2006 24:00:00 186 million 4,713 MB
20061011-000000-0 Wed Oct 11 13:00:01 2006 24:00:00 186 million 4,775 MB
20061012-000000-0 Thu Oct 12 13:00:01 2006 24:00:00 180 million 4,615 MB
20061013-000000-0 Fri Oct 13 13:00:01 2006 24:00:00 128 million 3,263 MB
20061014-000000-0 Sat Oct 14 13:00:01 2006 11:00:02 53 million 1,327 MB
20061014-110002-1 Sun Oct 15 00:00:02 2006 12:59:58 50 million 1,290 MB
20061015-000000-0 Sun Oct 15 13:00:01 2006 24:00:00 142 million 3,614 MB
20061016-000000-0 Mon Oct 16 13:00:01 2006 24:00:00 180 million 4,695 MB
20061017-000000-0 Tue Oct 17 13:00:01 2006 24:00:00 174 million 4,478 MB
20061018-000000-0 Wed Oct 18 13:00:01 2006 24:00:00 175 million 4,523 MB
20061019-000000-0 Thu Oct 19 13:00:01 2006 24:00:00 175 million 4,482 MB
20061020-000000-0 Fri Oct 20 13:00:01 2006 24:00:00 134 million 3,430 MB
20061021-000000-0 Sat Oct 21 13:00:01 2006 11:00:03 53 million 1,339 MB
20061021-110003-1 Sun Oct 22 00:00:03 2006 12:59:57 43 million 1,124 MB
20061022-000000-0 Sun Oct 22 13:00:01 2006 24:00:00 110 million 2,797 MB
20061023-000000-0 Mon Oct 23 13:00:01 2006 24:00:00 155 million 3,991 MB
20061024-000000-0 Tue Oct 24 13:00:01 2006 24:00:00 171 million 4,389 MB
20061025-000000-0 Wed Oct 25 13:00:01 2006 24:00:00 163 million 4,159 MB
20061026-000000-0 Thu Oct 26 13:00:01 2006 24:00:00 167 million 4,237 MB
20061027-000000-0 Fri Oct 27 13:00:01 2006 24:00:00 124 million 3,147 MB
20061028-000000-0 Sat Oct 28 13:00:01 2006 11:00:02 50 million 1,259 MB
20061028-110002-1 Sun Oct 29 00:00:02 2006 12:59:58 40 million 1,023 MB
20061029-000000-0 Sun Oct 29 13:00:01 2006 24:00:00 123 million 3,139 MB
20061030-000000-0 Mon Oct 30 13:00:01 2006 24:00:00 173 million 4,398 MB
20061031-000000-0 Tue Oct 31 13:00:01 2006 24:00:00 162 million 4,145 MB
20061101-000000-0 Wed Nov 1 13:00:01 2006 24:00:00 172 million 4,439 MB
20061102-000000-0 Thu Nov 2 13:00:01 2006 24:00:00 162 million 4,183 MB
20061103-000000-0 Fri Nov 3 13:00:01 2006 24:00:00 113 million 2,882 MB
20061104-000000-0 Sat Nov 4 13:00:01 2006 11:00:02 42 million 1,080 MB
20061104-110003-1 Sun Nov 5 00:00:03 2006 12:59:58 38 million 984 MB
20061105-000000-0 Sun Nov 5 13:00:01 2006 24:00:00 123 million 3,121 MB
20061106-000000-0 Mon Nov 6 13:00:01 2006 24:00:00 164 million 4,203 MB
20061107-000000-0 Tue Nov 7 13:00:01 2006 24:00:00 156 million 3,978 MB
20061108-000000-0 Wed Nov 8 13:00:01 2006 24:00:00 153 million 3,926 MB
20061109-000000-0 Thu Nov 9 13:00:01 2006 24:00:00 180 million 4,588 MB
20061110-000000-0 Fri Nov 10 13:00:01 2006 24:00:00 113 million 2,882 MB
20061111-000000-0 Sat Nov 11 13:00:01 2006 11:00:03 42 million 1,094 MB
20061111-110003-1 Sun Nov 12 00:00:03 2006 12:59:57 41 million 1,070 MB
20061112-000000-0 Sun Nov 12 13:00:01 2006 24:00:00 124 million 3,208 MB
20061113-000000-0 Mon Nov 13 13:00:01 2006 24:00:00 167 million 4,338 MB
20061114-000000-0 Tue Nov 14 13:00:01 2006 24:00:00 146 million 3,750 MB
20061115-000000-0 Wed Nov 15 13:00:01 2006 24:00:00 154 million 3,961 MB
20061116-000000-0 Thu Nov 16 13:00:01 2006 24:00:00 155 million 3,973 MB
20061117-000000-0 Fri Nov 17 13:00:01 2006 24:00:00 116 million 2,989 MB
20061118-000000-0 Sat Nov 18 13:00:01 2006 11:00:03 41 million 1,055 MB
20061118-110003-1 Sun Nov 19 00:00:03 2006 12:59:57 40 million 1,051 MB
20061119-000000-0 Sun Nov 19 13:00:01 2006 24:00:00 116 million 2,981 MB
20061120-000000-0 Mon Nov 20 13:00:01 2006 24:00:00 161 million 4,161 MB
20061121-000000-0 Tue Nov 21 13:00:01 2006 24:00:00 152 million 3,912 MB
20061122-000000-0 Wed Nov 22 13:00:01 2006 24:00:00 153 million 3,935 MB
20061123-000000-0 Thu Nov 23 13:00:01 2006 24:00:00 129 million 3,280 MB
20061124-000000-0 Fri Nov 24 13:00:01 2006 24:00:00 96 million 2,431 MB
20061125-000000-0 Sat Nov 25 13:00:01 2006 11:00:03 37 million 939 MB
20061125-110003-1 Sun Nov 26 00:00:03 2006 12:59:57 54 million 1,391 MB
20061126-000000-0 Sun Nov 26 13:00:01 2006 24:00:00 108 million 2,817 MB
20061127-000000-0 Mon Nov 27 13:00:01 2006 24:00:00 177 million 4,638 MB
20061128-000000-0 Tue Nov 28 13:00:01 2006 24:00:00 187 million 4,927 MB
20061129-000000-0 Wed Nov 29 13:00:01 2006 24:00:00 141 million 3,683 MB
20061130-000000-0 Thu Nov 30 13:00:01 2006 24:00:00 174 million 4,554 MB
20061201-000000-0 Fri Dec 1 13:00:01 2006 24:00:00 147 million 3,737 MB
20061202-000000-0 Sat Dec 2 13:00:01 2006 11:00:02 54 million 1,332 MB
20061202-110003-1 Sun Dec 3 00:00:03 2006 12:59:58 55 million 1,378 MB
20061203-000000-0 Sun Dec 3 13:00:01 2006 24:00:00 128 million 3,279 MB
20061204-000000-0 Mon Dec 4 13:00:01 2006 24:00:00 145 million 3,744 MB
20061205-000000-0 Tue Dec 5 13:00:01 2006 24:00:00 133 million 3,464 MB
20061206-000000-0 Wed Dec 6 13:00:01 2006 24:00:00 171 million 4,529 MB
20061207-000000-0 Thu Dec 7 13:00:01 2006 24:00:00 149 million 3,891 MB
20061208-000000-0 Fri Dec 8 13:00:01 2006 24:00:00 131 million 3,392 MB
20061209-000000-0 Sat Dec 9 13:00:01 2006 11:00:02 36 million 928 MB
20061209-110002-1 Sun Dec 10 00:00:02 2006 12:59:58 36 million 952 MB
20061210-000000-0 Sun Dec 10 13:00:01 2006 24:00:00 106 million 2,712 MB
20061211-000000-0 Mon Dec 11 13:00:01 2006 24:00:00 151 million 3,906 MB
20061212-000000-0 Tue Dec 12 13:00:01 2006 24:00:00 139 million 3,598 MB
20061213-000000-0 Wed Dec 13 13:00:01 2006 24:00:00 136 million 3,511 MB
20061214-000000-0 Thu Dec 14 13:00:01 2006 24:00:00 128 million 3,298 MB
20061215-000000-0 Fri Dec 15 13:00:01 2006 24:00:00 95 million 2,435 MB
20061216-000000-0 Sat Dec 16 13:00:01 2006 11:00:02 31 million 768 MB
20061216-110003-1 Sun Dec 17 00:00:03 2006 12:59:58 33 million 865 MB
20061217-000000-0 Sun Dec 17 13:00:01 2006 24:00:00 103 million 2,638 MB
20061218-000000-0 Mon Dec 18 13:00:01 2006 24:00:00 170 million 4,331 MB
20061219-000000-0 Tue Dec 19 13:00:01 2006 24:00:00 146 million 3,780 MB
20061220-000000-0 Wed Dec 20 13:00:01 2006 24:00:00 144 million 3,699 MB
20061221-000000-0 Thu Dec 21 13:00:01 2006 24:00:00 153 million 3,794 MB
20061222-000000-0 Fri Dec 22 13:00:01 2006 24:00:00 93 million 2,301 MB
20061223-000000-0 Sat Dec 23 13:00:01 2006 11:00:02 38 million 924 MB
20061223-110003-1 Sun Dec 24 00:00:03 2006 12:59:58 39 million 982 MB
20061224-000000-0 Sun Dec 24 13:00:01 2006 24:00:00 77 million 1,909 MB
20061225-000000-0 Mon Dec 25 13:00:01 2006 24:00:00 74 million 1,880 MB
20061226-000000-0 Tue Dec 26 13:00:01 2006 24:00:00 75 million 1,916 MB
20061227-000000-0 Wed Dec 27 13:00:01 2006 24:00:00 73 million 1,837 MB
20061228-000000-0 Thu Dec 28 13:00:01 2006 24:00:00 77 million 1,947 MB
20061229-000000-0 Fri Dec 29 13:00:01 2006 24:00:00 68 million 1,715 MB
20061230-000000-0 Sat Dec 30 13:00:01 2006 11:00:02 26 million 639 MB
20061230-110003-1 Sun Dec 31 00:00:03 2006 12:59:58 28 million 715 MB
20061231-000000-0 Sun Dec 31 13:00:01 2006 24:00:00 57 million 1,395 MB
20070101-000000-0 Mon Jan 1 13:00:01 2007 24:00:00 56 million 1,386 MB
20070102-000000-0 Tue Jan 2 13:00:01 2007 24:00:00 89 million 2,208 MB
20070103-000000-0 Wed Jan 3 13:00:01 2007 24:00:00 113 million 2,831 MB
20070104-000000-0 Thu Jan 4 13:00:01 2007 24:00:00 114 million 2,870 MB
20070105-000000-0 Fri Jan 5 13:00:01 2007 24:00:00 104 million 2,660 MB
20070106-000000-0 Sat Jan 6 13:00:01 2007 11:00:03 37 million 957 MB
20070106-110003-1 Sun Jan 7 00:00:03 2007 12:59:57 41 million 1,061 MB
20070107-000000-0 Sun Jan 7 13:00:01 2007 24:00:00 105 million 2,650 MB
20070108-000000-0 Mon Jan 8 13:00:01 2007 24:00:00 200 million 5,073 MB
20070109-000000-0 Tue Jan 9 13:00:01 2007 24:00:00 238 million 6,199 MB
20070110-000000-0 Wed Jan 10 13:00:01 2007 24:00:00 207 million 5,363 MB
20070111-000000-0 Thu Jan 11 13:00:01 2007 24:00:00 166 million 4,219 MB
20070112-000000-0 Fri Jan 12 13:00:01 2007 24:00:00 121 million 3,078 MB
20070113-000000-0 Sat Jan 13 13:00:01 2007 11:00:02 40 million 1,016 MB
20070113-110003-1 Sun Jan 14 00:00:03 2007 12:59:58 46 million 1,212 MB
20070114-000000-0 Sun Jan 14 13:00:01 2007 24:00:00 121 million 3,088 MB
20070115-000000-0 Mon Jan 15 13:00:01 2007 24:00:00 160 million 4,067 MB
20070116-000000-0 Tue Jan 16 13:00:01 2007 24:00:00 146 million 3,724 MB
20070117-000000-0 Wed Jan 17 13:00:01 2007 24:00:00 148 million 3,693 MB
20070118-000000-0 Thu Jan 18 13:00:01 2007 24:00:00 150 million 3,660 MB
20070119-000000-0 Fri Jan 19 13:00:01 2007 24:00:00 105 million 2,628 MB
20070120-000000-0 Sat Jan 20 13:00:01 2007 11:00:02 37 million 937 MB
20070120-110003-1 Sun Jan 21 00:00:03 2007 12:59:58 40 million 1,018 MB
20070121-000000-0 Sun Jan 21 13:00:01 2007 24:00:00 110 million 2,744 MB
20070122-000000-0 Mon Jan 22 13:00:01 2007 24:00:00 158 million 4,000 MB
20070123-000000-0 Tue Jan 23 13:00:01 2007 24:00:00 152 million 3,775 MB
20070124-000000-0 Wed Jan 24 13:00:01 2007 24:00:00 160 million 4,026 MB
20070125-000000-0 Thu Jan 25 13:00:01 2007 24:00:00 156 million 3,959 MB
20070126-000000-0 Fri Jan 26 13:00:01 2007 24:00:00 111 million 2,826 MB
20070127-000000-0 Sat Jan 27 13:00:01 2007 11:00:03 39 million 995 MB
20070127-110003-1 Sun Jan 28 00:00:03 2007 12:59:57 42 million 1,091 MB
20070128-000000-0 Sun Jan 28 13:00:01 2007 24:00:00 80 million 2,010 MB
20070129-000000-0 Mon Jan 29 13:00:01 2007 24:00:00 125 million 3,179 MB
20070130-000000-0 Tue Jan 30 13:00:01 2007 24:00:00 172 million 4,329 MB
20070131-000000-0 Wed Jan 31 13:00:01 2007 24:00:00 161 million 4,125 MB
20070201-000000-0 Thu Feb 1 13:00:01 2007 24:00:00 156 million 3,971 MB
20070202-000000-0 Fri Feb 2 13:00:01 2007 24:00:00 124 million 3,183 MB
20070203-000000-0 Sat Feb 3 13:00:01 2007 11:00:02 50 million 1,259 MB
20070203-110003-1 Sun Feb 4 00:00:03 2007 12:59:58 40 million 1,039 MB
20070204-000000-0 Sun Feb 4 13:00:01 2007 24:00:00 118 million 2,971 MB
20070205-000000-0 Mon Feb 5 13:00:01 2007 24:00:00 120 million 3,005 MB
20070206-000000-0 Tue Feb 6 13:00:01 2007 24:00:00 122 million 3,066 MB
20070207-000000-0 Wed Feb 7 13:00:01 2007 24:00:00 153 million 3,859 MB
20070208-000000-0 Thu Feb 8 13:00:01 2007 14:24:51 106 million 2,696 MB
20070211-213630-0 Mon Feb 12 10:36:30 2007 2:23:30 32 million 841 MB
20070212-000000-0 Mon Feb 12 13:00:01 2007 24:00:00 174 million 4,470 MB
20070213-000000-0 Tue Feb 13 13:00:01 2007 24:00:00 170 million 4,299 MB
20070214-000000-0 Wed Feb 14 13:00:01 2007 24:00:00 161 million 4,061 MB
20070215-000000-0 Thu Feb 15 13:00:01 2007 24:00:00 155 million 3,913 MB
20070216-000000-0 Fri Feb 16 13:00:01 2007 24:00:00 109 million 2,732 MB
20070217-000000-0 Sat Feb 17 13:00:01 2007 11:00:02 35 million 887 MB
20070217-110003-1 Sun Feb 18 00:00:03 2007 12:59:58 35 million 905 MB
20070218-000000-0 Sun Feb 18 13:00:01 2007 24:00:00 122 million 3,058 MB
20070219-000000-0 Mon Feb 19 13:00:01 2007 24:00:00 161 million 4,076 MB
20070220-000000-0 Tue Feb 20 13:00:01 2007 24:00:00 154 million 3,896 MB
20070221-000000-0 Wed Feb 21 13:00:01 2007 24:00:00 148 million 3,776 MB
20070222-000000-0 Thu Feb 22 13:00:01 2007 24:00:00 152 million 3,833 MB
20070223-000000-0 Fri Feb 23 13:00:01 2007 24:00:00 124 million 3,181 MB
20070224-000000-0 Sat Feb 24 13:00:01 2007 11:00:02 46 million 1,183 MB
20070224-110003-1 Sun Feb 25 00:00:03 2007 12:59:58 45 million 1,186 MB
20070225-000000-0 Sun Feb 25 13:00:01 2007 24:00:00 133 million 3,425 MB
20070226-000000-0 Mon Feb 26 13:00:01 2007 24:00:00 186 million 4,768 MB
20070227-000000-0 Tue Feb 27 13:00:01 2007 24:00:00 196 million 4,989 MB
20070228-000000-0 Wed Feb 28 13:00:01 2007 24:00:00 181 million 4,633 MB
20070301-000000-0 Thu Mar 1 13:00:01 2007 24:00:00 170 million 4,268 MB
20070302-000000-0 Fri Mar 2 13:00:01 2007 24:00:00 132 million 3,364 MB
20070303-000000-0 Sat Mar 3 13:00:01 2007 11:00:02 53 million 1,336 MB
20070303-110003-1 Sun Mar 4 00:00:03 2007 12:59:58 45 million 1,168 MB
20070304-000000-0 Sun Mar 4 13:00:01 2007 24:00:00 156 million 3,962 MB
20070305-000000-0 Mon Mar 5 13:00:01 2007 24:00:00 232 million 6,004 MB
20070306-000000-0 Tue Mar 6 13:00:01 2007 24:00:00 206 million 5,222 MB
20070307-000000-0 Wed Mar 7 13:00:01 2007 24:00:00 217 million 5,602 MB