An update to the CDF analysis at the 99.99% joint confidence MDA data was required. When the multipath detection algorithm runs sometimes flow IDs are repeated to test for per packet load balancers. It was necessary to take these repeats out of the analysis that estimates stopping values from real world Internet data.
Another change to this analysis was to include comparison between doing the analysis with and without allowing repeats of the same load balancing interface. On one hand it is desirable to avoid repeated analysis of the same part of the Internet for sample integrity, and on the other hand processing larger numbers of load balancers to produce CDF data allows for more data points beyond the cut off probability, a desirable outcome.
I have made updates to the chapter on stopping values in my thesis, based on Richards critique.
Continued working on adding new rules to libprotoident based on unknown flows seen with the new Waikato capture. Since getting access to fresh traffic, I've added 12 new protocols and improved the rules for another 13 existing ones.
Some of the more notable protocols that I've added are QUIC, SPDY, WeChat, Git and Speedtest. Also added a rule for the AMP throughput test, as this is one of the biggest contributors of "Unknown" traffic.
Captured a full weekday of traffic to use as a basis for working out how regularly we can take permanent captures and what sort of duration we can reasonably expect to capture for. A single day is around 116 GB (snapped and compressed). To put this in context, ~100 days of similar capture from 2007 was 491 GB -- a little over 4 days worth of traffic now.
The security supplicant is now functional - encryption and decryption. All devices are using the same key which is written at compile time. I need to do some research into viable key exchange mechanisms, but I figure something like 802.11i / 802.11ai based will do the trick. 802.15.9 documents a key exchange protocol, however only members of IEEE 802 WG4 are able to see it. There is a very brief draft which I might be able to use to get something similar in functionality.
There's actually a fair chunk of functionality missing from the security supplicant. It is missing support for other key ID modes (explicit) as well as different security levels (MIC only, ENC with 64 and 128 bit MICs). It should be easy to add the additional security levels, however I think that this is a nice to have and doesn't really affect the end result.
My final presentation is not going to be on the 2nd of September. I am attending a wedding in Germany so my seminar will instead be on the 18th of August at 11am. More information to follow on the department mailing list!
Soon I will have to start focussing on the presentation rather than getting new features implemented.
Built a de/muxer for VLANs. The only thing I don't like is if I attach my RADIUS port to a low port number, say, port 2, SVID 10 traffic will come out on port 11.
To counter this, I might statically reserve "special" ports, and then offset the incoming ports by that number i.e. reserve the first 4, and then anything I see come in will be off by 4. Doing this statically is a little tedious, but it's better than it not working at all.
Rewrote some of the code around ASN lookups for traceroute tests to make it more robust in the case of the server being unreachable. Failure to connect is now detected much quicker (using non-blocking sockets) and if anything goes wrong with a remote lookup then the thread will stop trying and respond only with cached data. The flow of control is now also a lot simpler, which means sockets always get read from in a timely manner and the code is a lot clearer.
Spent longer than I would have liked trying to diagnose the problems around ASN lookups due to not realising rsyslog was throttling all the extra debug output I had added.
Had a bit of a deeper look into some of the long duration HTTP tests to see if there might be any patterns of interest. Trademe stands out as the site that has the most slow (>10 second) page fetches, but the majority of them all come from the same connection, with other connections being perfectly fine. Most of the other targets are pretty consistent with each other.
Continued to work on oflops this week. I've worked through the code fixing many issues shown by valgrind. Such as correcting the order when stopping threads so that memory is no longer free'd by one thread while another is still accessing it. Along with many other memory leaks etc.
I've also looked at the kernel module packet generator (pktgen) which can be used to produce packets instead of using userspace pcap, and as expected it can produce higher packet rates than the userspace. Pktgen uses a delay between packets for rate limiting and will insert the current timestamp into a packet. However was applying the delay after the timestamping of the packet, this resulted in an offset in the timestamp similar to the delay however this is not guaranteed as the length of the delay varies depending upon how long tx and creating the next packet takes. I've rebuilt the module with the ordering fixed. When I get the time I would like submit my fix for this to the kernel.
More work on the dashboard this week:
* added the ability to remove "common" events from the recent event list and made the graphs collapsible.
* added a table that shows the most frequently occuring events in the past day, e.g. "increased latency from A to B (ipv4)".
* polished up some of the styling on the dashboard and moved the dashboard-specific CSS (of which there is now quite a lot) into its own separate file.
Started thinking about how to include loss-related events in the event groups, as these are ignored at the moment.
The new capture point came online on Wednesday, so the rest of my week was spent playing with the packet captures. This involved:
* learning to operate EndaceVision.
* installing wdcap on the vDAG VM.
* adding the ability to anonymise only the local network in wdcap.
* performing a short test capture.
* getting BSOD working again, which required the application of a little "in-flow" packet sampling to run smoothly.
* running libprotoident against the test capture to see what new rules I can add.
Defining the schema for network layer
- using RA with DNS backup instead of DHCP (stateless is easier to implement and less resource heavy)
- write up with a given setup what should happen when a new node connects to the gateway.
Moving onto application layer
- californium is a java CoAP API (more familiar with java)
- investigate what is possible with the API
- potentially look into dummy client / server setup on test machine (not the RPi's)
A modification was made to the diamond detection part of Megatree. This was a minor upgrade to better find the convergence points of asymmetric load balancer diamonds. In particular what was added, was a step to find the maximum value for probe TTL for a given link node. Doing this seems to reduce the chance of finding a node further downstream from the convergence interface as convergence. The updated analysis is running at present, and counts of convergence points found for a given divergence point will be compared with results before the modification.
I finished the initial checking of my thesis draft and so I am now hoping for some feedback to help finalise things.
I got AES going! 802.15.4 uses the CCM* block cipher mode with AES as the cipher (using 128bit keys).
This was really difficult! Small changes in the input resulted in wild changes in the output (as you would expect!). I wrote a tool with libtomcrypt to create some reference data (something that I would then try to replicate with the CC2538 libraries). In the end, I managed to make this work.
I also got this going with Wireshark (which highlighted an endianness issue with my libtomcrypt tool). Wireshark will now accept my AES encrypted messages which gives me confidence that the implementation is correct.
The implementation is a bit hacked together at the moment (as I was hacking about to get it going) however tidying it up should be simple. Then I can move on to decryption.
As for the security, I have no idea how well this implementation would perform against side-band power analysis attacks. This is something I'm not even going to consider trying to defend against at this stage! In the future, however, this might be important.