Added graphs for the HTTP test to amp-web, which helped reveal a couple of problems with the HTTP test that Brendon duly fixed.
Updated amp-web to support the new eventing database schema. Managed to get eventing up and running successfully, with significant event groups appearing on the dashboard and events also marked on the graphs themselves.
Gave my annual libtrace lectures, which seemed to go fairly well. Already I've got students thinking about and working on the assignment so they are at least smart enough to start early :)
Continued to develop a tutorial for my event classification app. Found plenty of good examples of different types of events so now it is just a matter of writing all the explanatory text for each example.
Spent most of this week doing other assignments to get on top of them early and out of the way, alongside reading material related to SDN (http://wand.net.nz/~brad/talks/sdn_outline.html). Had a good lecture from Brad which went over and explained what I had read. Also given an example program using Ryu to create a simple L2 switch, which should be a good starting point.
Submitted my 520 blurb to Bronwyn as required this week.
Spent some time checking over the results of the new schedule to make
sure all the targets were reporting sensible data. Found that for some
of the HTTP test targets it was reporting 302 codes, despite choosing
the target URLs specifically to avoid this. Turns out that I was
stripping the scheme portion and just passing the hostname to libcurl,
so URLs that should have been HTTPS were treated as HTTP. Fixed it so it
properly uses the full URL. Had to update some ports used in the tcpping
test too, in order to get a useful response.
Also updated the HTTP test to return more sensible data if the
connection times out, rather than storing the duration of the timeout
and flatlining the graphs, it now returns an empty duration which is
Wrote a custom init script and other config to deal with starting the
amplet client in a network namespace for the new test deployment.
Set up new scamper modifications on a test vantage point. The modes to be used are ICMP per destination limited to seven flow IDs and another mode with many flow IDs. Initially I encountered a bug but now the test run is on the way where both these modes are run for each destination address. This will allow comparison of the ability of these two methods to detect load balancers and compare their efficiencies at doing this task by comparing probe counts.
The churn paper has been updated to include the research on stopping values. It is not yet clear whether the simulation method to estimate stopping values should be included.
Updated the new test schedule slightly and set it running to collect
initial data. I'll look over that and make sure that all the targets are
responding as they should, and that the HTTP tests are fetching useful
pages (not redirects etc).
Got some useful looking stats from the embedded html5 youtube player
running with slimer.js inside a virtual frame buffer. Ideally it would
be fully headless, but it shows we should be able to get the information
we need from the player - initial buffering delay, other time spent
buffering, playing, etc.
Started a test install of the amplet components onto a machine using
network namespaces. Figured out what I need to do to make sure
everything can talk to each other in the right way. Also found and fixed
a few issues with the new PKI changes when used in a new environment
from scratch - some setup steps were relying on things that hadn't
happened yet and needed to be reordered.
20 new PlanetLab nodes where set up for using with the black hole detector. Once this was done a run was initiated.
Load balancers were counted in 6 runs that were previously carried out on PlanetLab. One of these used 128 flow IDs instead of the economical 7. This can probably be used for comparison even though they were carried out sequentially. The comparison is aimed at the number of load balancers found and whether more are found with the fuller analysis.
Changes from the TMA submission were applied to the churn paper. Preparation to apply to a new Journal also has meant reformatting to two columns again.
Continued working on my Django app for crowd-sourcing event classifications from the general public. The core app is functional but I found that it was very difficult to explain all the intricacies of my personal classification approach using textual instructions alone. As a result, I'm working on adding a tutorial component so that users can be trained using a series of practical exercises where they encounter most of the various types of events we typically see.
Updated eventing to be able to handle more than just latency events and hopefully group events across different collections, i.e. group traceroute path changes with changes in latency. Started testing eventing with live data, so hopefully our event dashboard shouldn't be too far away from being up and running again.
Ryan has asked me to help out with teaching libtrace in 513 again this semester. Spent a day or so working on a new assignment and updating my slides. Looking forward to seeing how the students go with this year's assignment, especially the MSS analysis task.
Fixed some minor edge cases that I found while repeatedly killing and
restarting various parts of the amplet client. Some strings describing
tests were invalidated when tests got reloaded, so are now stored. Some
sockets were able to be leaked if the control socket failed to start.
Tidied up the freeing of timers (tests, watchdogs, schedule updates) to
properly remove everything on exit. Also cleaned up the build process
and removed some extraneous flags, fixed some warnings and enabled
silent rules in automake.
Put together a new test schedule that covers more web targets (mostly
top ranked Alexa sites), as well as performing latency measurements to
some of the major gaming infrastructure targets. Took a while to find
some of these as they appear to be mostly undocumented.
Spent some time looking at measuring streaming video from youtube - the
embedded html5 player exposes some information that looks like it could
be used to make measurements. Currently trying to get it working with
the Zombie.js headless browser, but am stalled just before the video is
meant to start playing.
This week I deprecated the various statistic functions trace_get_filtered/accepted/dropped/received and replaced them with a single function trace_get_statistics().
This allows for a single atomic update of statistics, assuming the underlying format can provide this. I have also improved the documentation of the meaning of the statistics. Internally I removed the unused get_captured_packets interface.
I updated ring: and int: to use the new statistic interface and by using the device statistics managed to get some extra counters implemented. This included the number of packets filtered which was previously unavailable and the new packets errors.
I also updated the dag25: format to use the new interface as well as other minor fixes and testing of the dag format.
Results from a corrected run of the black hole in load balancer detector were processed. Some short traces were found that stopped in load balancers i.e. Discontinuities, were found. The detector will now be run on more nodes on PlanetLab to broaden the search.
A scamper routine that uses large numbers of destinations (256) for finding per destination load balancers was programmed. Also a driver to run the old routine and the new routine was programmed. A run of 5000 addresses is now underway on Yoyo. After this test a run will be carried out on my PlanetLab nodes.
More PlanetLab nodes have been activated on my slice. It is amazing how many do not come online when activated. This made it necessary to activate more on two occasions to get the numbers of usable nodes up to the required level of 25.