Put together a simple "infinitely scrolling" event list to allow browsing
of all historical events. Needs a bit of work to be properly useful, but
it was important to have the data available somehow so that it can be used
(and the way it is used can then lead to a better display approach).
Added some simple navigation buttons to change the zoom level of the
summary graph. For now, the right hand edge is fixed at the current time
and the amount of data shown in the summary can be increased or decreased.
Again this isn't an ideal interface, but does make the data available
and gives us a starting point to work from. Also fixed up some cosmetic
and small functionality issues in the display of graphs - gridlines are
now redrawn at the correct scale if the smokeping shading boosts the
length of the y-axis and areas in the summary without data can now be
Spent a bit more time working on packaging, getting initial Debianisation
done for netevmon (the event detection framework).
So I have everything working now, and have started writing the report. I should have a first draft of the first chapter finished today.
Some of the code is not as pretty as I would like it to be, I completely broke my interface between the load balancing module and rfserver..
Monitored the data collection on Caida and planetlab. Downloaded data as it became ready.
Data was downloaded from Caida (three teams) and from iplane (hubble like) and counts of accumulated interfaces were carried out on the former along with the same on the first run of my analysis data. The idea is to find out what data volume is required to get a reasonable coverage of the internet backbone and to determine if we can simulate it.
The internet simulator was started off in its absolute native state, and with longer abort times in a second run. I determined that the errors from the last run where next hops were not found were associated with simulations with many sources and gang analysis.
I've finished the C code now and I'm currently trying to use the Xilinx Vivado tool to convert it to VHDL. This is proving to be harder than I thought however I'm making good progress. At the moment I'll only running it on part of the code but once I'm comfortable with the tool I'll be able to convert my whole program. Hopefully only the FFT function will be a problem.
Added manpages to netevmon to get it ready for Debian packaging. During this process, fixed a few little oversights in the netevmon script and the existing documentation.
Re-wrote much of the NNTSC API in ampy. The main goal was to reduce the amount of duplicated code in modules for individual NNTSC collections that was better suited to a more general NNTSC API. In the process I also changed the API to only use a single "NNTSC Connection" instance rather than creating and destroying one for every AJAX request. The main benefit of this is that we don't have to ask the database about collections and streams every time we make a request now -- instead we get them once and store that info for subsequent use. This will hopefully make the graph interface feel a bit more responsive.
Updated amp-web to use the new NNTSC API in ampy. I also spent a bit of time on Friday testing the web graphs on various browsers and fixing a few of the more obvious problems. Unsurprisingly, IE 10 was the biggest source of grief.
Added a new time series type to anomaly_ts -- JitterVariance. This time series tracks the standard deviation of the latencies reported by the individual smokeping pings. Using this, I've added a new event type designed to detect when the standard deviation has moved away from being near zero, e.g. the pings have started reporting variable latency. This helps us pick up on situations where the median stays roughly the same but the variance clearly indicates some issues. It also serves as a good early indicator of upcoming Plateau or Mode events on the median latency.
Worked on researching whether there were existing OpenStack user management tools that met project requirements as well as what existing elements of OpenStack could be extended to fit project requirements for user management and control of instances. There did not appear to be any tools available that controlled the time period within which users could access and manipulate their instances.
Finished updating ndt-web10g, it now uses the newer library and supports logging. This means the full functionality included in ndt-web100 also works in ndt-web10g. This took a bit longer than I hoped - a few more issues showed up then I expected. This is currently sitting in the ndt-web10g branch, code.google.com/p/ndt/.
Started looking into improving libtrace's ability to capture packets at any speed without loss on commodity hardware (i.e. DAG card like performance). Found two solutions Intel Data Plane Development Kit (DPDK) and PF_RING Direct NIC Access (DNA).
Started looking at Intel DPDK, I'm yet to run anything besides sample applications. This comprises of a set of loadable kernel modules, meaning it can be built and loaded into a running kernel. And a new userland since switching a network interface to the DPDK module removes it from the kernel as a network interface.
This week I have created a new module to handle the forwarding between switches. This module at the moment is at a stage were it calculates the shortest path between switches and installs flows along that path. The module is pretty basic at the moment as it is just using hop count for the shortest path calculation, however next week I plan to introduce the links statistics into the calculation. After links stats are introduced I then plan to start looking at introducing load balancing into the module
Finished up the graphs for the simple dashboard to show when events took
place over the last day, and which sites/attributes were involved in the
largest number of events. More detailed information about event groups in
the last day are also shown, much as they were in the original dashboard
(expandable, with links to the graphs).
Got the initial debianisation done for NNTSC. Packages can now be built
that install all the files into the correct places, which should make
distributing it a lot easier - just a matter of correctly editing the
configuration file before starting (we can't guess this automatically).
Made a fix to the graphing library we are using to get selections working
properly. We had identified the problem a while ago, but with the bug
already reported and no fix coming from upstream decided to investigate
it. Following the suggested fix by the bug reporter appears to solve our
problems, with no obvious side effects.
Tidied up some AMP documentation describing the behaviour of schedules and
nametables, bringing it up to date with the current behaviour. Added
documented configuration files with some basic examples to the
Having bodly claimed to have finished debugging the path learning last week, when I tried to get it to put the paths onto switches, I found that it wasnt actually deleting the paths properly, it was just telling me that it had with how I was testing it.
So I ended up completely overhauling how I am storing these things, and am just working out the last few kinks in that now. But hopefully that means that when it comes time to actually use them the whole process should be a lot easier.. Hopefully..