Configured the third throughput test target and updated the test
schedule to properly include all three throughput test targets. Went
through all the results to make sure that all are reporting - found and
fixed a couple where incorrect HTTP targets had been set and redirects
were happening. Double checked that some unusual throughput results were
correct (they appear to be).
Spent some time investigating some connections that appeared to up, but
wouldn't forward my data. The modem seems to think everything is fine
and there is nothing obviously wrong at my end, so I've asked for it to
Officially started my PhD this month, 1st April. As such I've switched from focusing on libtrace to software defined networking. With short weeks due to Easter, this report is covering the first two weeks of April.
I've started reading over some previous research of WAND students and other useful resources that Brad has directed me to.
With Brad's assistance I've also setup a virtual machine running a controller, running RouteFlow --- essentially a Vandervecken deployment. This connects to a mininet VM where an Open vSwitch is acting as the OpenFlow controllable device.
I've meet with Richard Nelson and he's suggested adding 'fastpath' to RouteFlow as a first project to get a feel for SDN development and break up the task of researching literature.
Packets that need to be processed by the controller are currently sent via a OpenFlow PACKET-IN message and if necessary returned to the via a PACKET-OUT message. There is a lot of overhead involved in this process, including adding extra meta data to a packet. Fastpath aims to eliminate this by using the data plane to forward packets to the controller quickly by adding OpenFlow rules. One difficulty is attaching the meta-data, i.e. the switch and port number.
I've also meet with Bill Rogers. He wanted to see some good specific use cases for SDN, particularly those which cannot be solved by the current technologies rather than the argument SDN gives people to flexibility to do whatever they want. Also while SDN can make many things easier many problems can be solved by some combination of current technologies.
The Internet simulator IS0 was tasked to do some strategic simulations. One of the angles is varying control packet queuing time for the sources window setting of 500. There are also a couple of control runs for some different initial TTL settings.
An attempt has been made to reduce the amount of extra probing required by the Megatree simulator. Currently a full simple Traceroute run is required to determine if a Traceroute contains a divergence convergence point pair that has been seen before. This is being reduced to do some extra probing when a previously seen divergence point is encountered in the main MDA trace.
I have printed my entire thesis draft and have started proof reading it from the beginning. This is an attempt to ensure that a well rounded story is told with good logical flow and appropriate discussion.
Short week again with the Easter break. Finished the setup of the second
measurement machine and the central collector/graphing machine. Got the
measurement machine shipped, which should hopefully be physically
installed in a few days.
Added some temporary throughput tests to the main schedule to test
performance across different times of day to each of the sites. The
proper schedule will need to be slightly tweaked to include the extra
throughput targets. So far the test data shows the targets to be
Went to Auckland with Brad to install a new measurement machine. It's
now up and running and performing a subset of tests, but generally looks
to be doing the right thing. Will be watching the data and adding more
tests over the next little while.
Performed some testing on the first throughput targets that we have
available to make sure that they will be fast enough. Two out of the
three so far look good, but the third is probably not well connected
enough to push the amount of data we are expecting.
Started to install a second measurement machine based on the first, as
well as documenting parts of the process that will need to be performed
by remote hands.
The first few weeks I focused on the blurb and proposal.
I found a trace file and wrote a script to output Python code which I piped into the Python terminal to save it to a SQLite database. I then queried the data in Django and graphed some information including protocol counts and a usage timeline.
Last week I discovered Shane's Libprotoident library for application layer protocol identification for flows. I thought it would be great if I could utilise this to display application information for the flows I will collect. I went to see Shane, who told me a previous students project used Libprotoident to request the type of information I would like. I will look into this over the next few weeks.
This week I have been busy with assignments. I hope to continue with the web interface early next week.
Spent this week working on other assignments to get them finished before end of teaching recess and didn't get to work on project.
Next step is still to research popular document learning algorithms and if they exist within Mallet see how well they adapt to log files, adjusting them if needed for a better fit to this text format.
Still fighting with cross compiling a workable linux kernel for RaspberryPi
Through different tutorials for compiling and running custom kernels and configuring u-boot managed to get a very unstable (laggy) system up n running.
Continued hunting for the bug in the NNTSC live exporter with mixed success. I've narrowed it down to definitely being the per-client queue that is the problem and it doesn't appear to be due to any obvious slowness inserting into the queue. Unfortunately, the problem seems to only occur once or twice a day so it takes a day before any changes or additional debugging take effect.
Went back to working on the mechanical Turk app for event detection. Finally finished a tutorial that shows most of the basic event types and how to classify them properly. Got Brendon and Brad to run through the tutorial and tweaked it according to their feedback. The biggest problem is the length of the tutorial -- it takes a decent chunk of our survey time to just run through the tutorial so I'm working on ways to speed it up a bit (as well as event classification in general). These include adding hot-keys for significance rating and using an imagemap to make the "start time" graph clickable.
Spent a decent chunk of my week trying to track down an obscure libtrace bug that affected a couple of 513 students, which would cause the threaded I/O to segfault whenever reading from the larger trace file. Replicating the bug proved quite difficult as I didn't have much info about the systems they were working with. After going through a few VMs, I eventually figured out that the bug was specific to 32 bit little-endian architectures: due to some lazy #includes, the size of an off_t was either 4 to 8 bytes between different parts of the libwandio source code which resulted in some very badly sized reads. The bug was found and fixed a bit too late for those affected students unfortunately.
Since the last report I've been reading through the 802.15.4 standard to get an idea of what to expect from silicon radios that say they are compliant.
Both Dean and Richard made the point that just because they use the 802.15.4 radios doesn't mean they follow the standard and as such I shouldn't expect to see a whole lot of structure from the standard. For example the standard is very vague when referring to a network not using PAN controller beacons which would allow the network designer to make whatever decisions they like. This is an issue when interoperability is desired as the IEEE802.15.4 standard really doesn't offer anything that would facilitate such interoperability - the application and the networking (in cases where it is not a star network with PAN controller beacons enables) are not defined by the standard.
I also attempted to trace through the source for both RIOT OS and ContikiOS. RIOT OS is a microkernel meaning that messages get sent between different processes by the kernel. As such, I was able to trace the transmission of a TCP packet through the RPL routing, through the 802.15.4 framing to a call where the frame was sent to a transceiver. ContikiOS was far more complicated as they use a lot of preprocessor macros for various things which makes tracing a little more difficult. I wasn't interested in spending a lot of time tracing when my main interest was just figuring out how the 15.4 layer worked (PAN beacons? PAN controller elections? free for all?). I actually didn't figure out how either worked, but I imagine reading through the RPL routing document would give more information.
Finally I tried to evaluate both ContikiOS and RIOT OS. I like the RIOT OS source as it is very clean and readable. For this reason I was really hoping that I could use RIOT OS for this project. However, support for the CC2538 transceiver (distinct from the microcontroller core) is not in the mainline branch. I was able to checkout a branch that claimed support (and is in the process of a pull request) however it was hard to tell if it was actually working. ContikiOS on the other hand was ready to go out of the box. My sniffer seemed to have some kind of latency in showing me received packets (or it was rejecting them because they aren't using standard frame checksums?) which made it difficult to figure out how packets were flowing. In any case I was able to create a gateway running SLIP to a tap interface as well as a web server node that I could access over the air using the gateway.
I plan to continue evaluating both options but it appears that ContikiOS is more polished than RIOT OS in terms of CC2538 support. I find it interesting that RIOT OS claims support for the CC2538DK but doesn't support the radio (yet). RIOT OS is an embedded RTOS but I don't think I would choose it over FreeRTOS unless I wanted the networking and radio stack.