Spent the week doing more reading into possible detectors, and read a couple of related surveys that compared different anomaly detection techniques. A lot of them seemed inappropriate for Netevmon, unfortunately. Did find some Bayesian methods that might be relevant and possible to implement.
Next week: investigate T-entropy and find out how it could be implemented within Netevmon
The results from the fields-driver-one scamper run on Caida are being downloaded. This explores non load balancing fields for possible rare load balancing influence.
The results from the scamper run3 have been computed and tabulated in conjunction with runs 1 & 2. These runs measure turnover of load balancers.
A plan for version control of the Internet Simulator has been devised, as the unzipped programs are not active svn copies.
Following on from the authentication work last week allowing AMP to auth
against a remote rabbitmq broker using SSL, I configured a local broker
to do the same. Using a local broker to initially receive results adds
reliability and means results won't get lost if the remote server is
unavailable for any reason. The way the SSL authentication and user
validation works means that even though the measurement client and
broker are out of our control, they can't falsify their identity to
report results as another user.
Started packaging up new AMP and some of the dependencies for CentOS and
Debian. Unfortunately adding dependencies has the problem that every
packaging system has out of date versions. Put together a patch for
librabbitmq-c with my changes for EXTERNAL auth. Will have a look at it
again to make sure I'm being sensible and then see if I can get it
Spent most of Wednesday listening to student practice talks ahead of the
honours conference. The overall quality of the presentations was quite
high, and they should do even better at the real presentations with a
week of polish and practice.
Added a smarter method of generating tick labels on the X axis to amp-web. Previously, if you were zoomed in far enough, the labels simply showed a time with no indication as to what day you are looking at. Now, we show the date as well as the time.
Reworked how zoom behaviour works with the summary graph. The zoom-level is now determined dynamically based on the selected range, e.g. selecting more than 75% of the current summary range will cause it to zoom out to the next level. Selecting a small area will cause it to zoom back in.
To support arbitrary changes to the summary graph range without having to re-fetch and re-draw both graphs, I decided to rewrite our graph management scripts to operate on an instance of a class rather than just being a function that gets called whenever we want to render the graphs. The class has methods that update just the summary graph or just the detail graph, so we only end up changing the graph that we need to. Also, the class can be subclassed to support different graph styles easily, e.g. our Smokeping style. While I was rewriting, I used jQuery.when to make all of the AJAX requests for graph data simultaneously rather than sequentially as we were previously.
This week much of my time was spent on my presentation an other assignments. I expect next week will be much the same
Worked on conference presentation. Created a prototype booking and user account creation request system that doesn't link directly with OpenStack API yet. Working on making the design look more cohesive with the Horizon dashboard before the conference on Wednesday.
So this week I have worked on my conference presentation and read a lot of stuff about fault detection in and outside of SDN.
The SDN stuff seems to mostly agree that to be fast you need fast-failover groups, which arent in open vswitch yet.
For non sdn stuff I read stuff on EOAM and BFD and RFC 6374. As well as some other random stuff that wasnt much use.
Downloading of warts data from Caida has been carried out.
Analysis of research data has been extended to include the search for load balancer nodes that no longer have more than one successor, but that did on another occasion.
A driver for testing non load balancing fields for load balancing behaviour has been written and tested. The job was divided into two drivers so that there are only six or seven short traces at a time.
Wrote a proposal for the project, so as to get a better understanding and also formalise what I broadly need to focus on. Fell sick on Tuesday morning and it got worse by the evening, so was not able to get any work done for the rest of the week.
Next week should still involve more reading and looking at possible methods that could be used in new detectors.
Tidied up the code I hacked in the previous week for displaying AMP ICMP
data on a smokeping style graph. There were a few edge cases around
missing test responses that were causing extra NULLs to be added to the
list of results. The database should now remove those before any other
stage sees the data, which simplifies the later code a lot. Also had to
make a few changes to how the smokeping loss data is presented to make
it match the AMP ICMP data, allowing them to use the same colouring.
Fixed the new AMP traceroute test to no longer report extraneous
unresponsive hops on the end of the path. While trying to shortcut a lot
of extra testing on incoming ICMP packets, it was being a bit too
accepting of TTL exceeded messages which was resetting some counters.
Sat down with the rabbitmq documentation and had a good think about the
best way to do authentication for AMP. I knew it could do SSL but wasn't
sure how that could tie in with rabbitmq/AMP users and ensure users
couldn't report data for others (in the situation where we don't have
total control over all the monitors). Digging deeper, the rabbitmq
server can authenticate against a list of users based on parts of the
client certificate (common name, a few others) - this means the username
is embedded in the certificate, the username must exist on the server
and the server must trust the cert.
After updating the rabbitmq-c library to the newest version to get SSL
support (and editing it to enable advertising some extra authentication
mechanisms) I now have a client that establishes an SSL connection to
the server using an "EXTERNAL" authentication mechanism. The server
treats the common name as the username to login with, and validates that
the userid in each message matches the one from the cert.