User login

Blogs

12

Aug

2013

Spent another couple of days moving code around in amp-web to make it tidier and easier to work with. Hopefully, Brendon will still be able to find things inside the codebase...

Added support for the amp-traceroute collection to amp-web. The graph is just a placeholder at the moment (a line graph of hop counts) until we get around to implementing the more useful stacked hop count graph using envision.

Re-enabled the tabs on the right-hand side of the graphs that allowed switching between related graphs, albeit without the preview graphs that used to be on them. The original tabs were very AMP-specific and hard-coded to appear on every graph. Now, the tabs are generated dynamically by an AJAX request that asks ampy for a list of "related" streams to the one currently being displayed. For example, an LPI byte count stream would have tabs showing flow, packet and user counts for the same source and application protocol whereas AMP streams will have tabs showing latency and traceroute for the same source-destination pair.

To avoid page reloads when using the tabs to switch between collections, I changed the dropdowns to be generated dynamically via an AJAX request rather than being placed and populated via the python run when the page is loaded.

12

Aug

2013

I've nearly got the FFT working in hardware with a device driver and system call for linux so I can just swap out the fft function for the hardware version in my code for performance testing. It has been a mission getting this far with so many little issues occurring which take ages to debug and fix.

11

Aug

2013

Researched various ways of either integrating or communicating bewteen the booking database and the OpenStack nova database. Drew activity and state diagrams to help with the design of the booking system.

09

Aug

2013

Put the rest my DPDK format documentation on the libtrace wiki.

Started looking at rrdbot, setup a copy to listen to variables. Started looking through the code and estimated the difficultly of a few modifications REANNZ wanted done.

09

Aug

2013

Finished off the script that generates random topologies this week. Next I want to create a third controller that pre-computes the minimum paths then uses them to forward, similar to the interface based controller I already have. From there will start looking at how the different controllers scale.

08

Aug

2013

Churn statistics were generated using counts of load balancers rather than diamonds. These were incorporated into my slides for the six monthly report. This was also to provide a reference point for verifying and improving the algorithm that I use to detect diamonds.

The six monthly report slides were presented.

The next round of scamper data collection on Caida and Planetlab was kicked off and monitored.

05

Aug

2013

Started the week by doing some reading and going over the theory behind a forecasting technique called Adaptive Response Rate Single Exponential smoothing. I then spent the next few days implementing and testing a detector that uses the smoothing technique to obtain the next forecasted byte count. Still need to figure out how to tweak the parameters so as to obtain a detector that does not produce a delayed copy of the actual measurements.

Plan for next week is to do some reading on event detection in time series data, especially looking at techniques/methods that NetEvMon does not currently use.

05

Aug

2013

Added traceroute data from AMP to the matrix display, which involved
feeding it through all stages of the pipeline - collection, parsing,
storage, querying. Expanded the matrix to be smarter about selecting the
collections to query and refactored some of the AMP data fetching code
to make it easier to add new AMP collections.

While doing testing with the data collected by the new AMP I found and
fixed a bug where the results of name resolution of some targets in the
schedule were not being properly used (only the first address was being
tested to). This has also shown that we need a sensible way to deal with
multiple targets having the same label in the matrix and graph displays.

Started looking at being able to format AMP ICMP test data in a similar
manner to smokeping so that it can be used with the more interesting
smokeping style graphs. Looks like the database can do most of the heavy
lifting to generate percentiles, which we can then use to plot the
shaded smokey regions.

Spent a bit of time updating the REANNZ weathermap to bring it in line
with recent network changes.

05

Aug

2013

Added support for the AMP ICMP collection to ampy and amp-web, so we are now able to plot graphs of the test data Brendon has been collecting.

Spent a decent chunk of an afternoon working through the DPDK build system with Richard S., trying to make the DPDK libraries build as position-independent code so that we can link libtrace against them nicely.

Reworked a large amount of code in amp-web to move the collection-specific code out of the core source files and into separate little modules for each collection. This means that the core code should be much easier to follow and work on. Adding support for new collections should also be simpler and require less inside knowledge of how the whole system works.

04

Aug

2013

Finished working on the autotools process for libtrace with DPDK so that '~make; make install' will build and install the shared library without any additional complexity. With Shane's assistance came to the conclusion that a patch is required to modify Intel's DPDK build system to allow libtrace to be built as a shared library.

Committed my work (i.e. the DPDK format) to the libtrace SVN. Put some basic documentation on the libtrace wiki (to be expanded upon).