The first half of this week I was exploring different tools that can be used for my project. Some of these tools included the tor browser for anonymity, a large range of Firefox add-ons preforming a range of functions such httpfox for viewing and modifying cookies and params submitted by the browser, and some low level network tools such as netcat and openssl for writing scripts to interact with the drive webservers through http requests on the command line.

The Second half of the week I began to scope my target google drive, I began collating the structure of the site, with infomation such as the urls and parameters used, as well as further research into web specific attacking techniques such as Cookie "Bit diddling"(Real term from Hacking Web Applications Exposed)

This week my main focus was to learn about the enviroment and tools I will be working in for my project.

Initially met up with Richard to discuss my project in a bit more depth to give me a better idea as to what I would be looking at doing. Also had a chat with Chris in order to find a bit more about openvswitch and ryu, how he has been using them and what sort of setup he would suggest for my project.

After the talk with Richard and Chris spent the rest of the week looking at setting an enviroment up for my project. This consited of an OVS switch and with some help from Brad some lxc containers to connect up to it.

Got the ryu simple_switch.py up and running

Spent the first half of the week working on the collector. Implemented exporting expired flow records and designed another protocol header and subheader for these records. Cleaned up some repetitive code and added a function to export the ongoing flow buffer when the timer expires(before checking for new ongoing flows). Also added some documentation.

Started working on the report in the middle of the week and so far, have a draft version of the first 4 chapters(excluding the intro). Shane has checked a couple of them already so the plan for the coming week is to tidy up those chapters and get as much writing done as possible.

The first true data collection cycle of planetlab has been completed and the data downloaded. The second cycle has been initiated and monitored. One of the nodes was wiped by the system before I could download the data it had collected. This change also manifested as the node having a different host key ID, which triggered a security warning from ssh.

Progress has been made in the area of dealiasing load balancers by analysing successor sets rather than load balancer IP address. In the first instance I plan to generate a new count of LBs and compare this with what I already have. Implementing this change to the collection of turnover statistics will involve a significant amount of programming.

I am hoping to get a little bit of guidance from Tony on how network data is presented to the internet simulator.

Spent the whole week going through AMP's various components, to get a feel of how it works. Have setup a basic dashboard application with customizable widgets which have a save state in the database to enable the same order on page refresh. Was also going through the API documentation of jQuery UI and Pyramid Pylons. I guess it will take a bit more time to get completely used to it and will try to run a sample graphing application suited to my needs by the end of this week.

Spent most of the week working through the new AMP graphing code to get a
feel for how it works. Had to refactor some portions and improve the
readability to properly understand what was going on. Also managed to
sneak in a few improvements to the loading times by removing queries for
data that wasn't really being used.

Installed the new amplet software onto our test machine to have a working
datasource for the event detection testing. Got it up and running and now
have to integrate the example consumer into NNTSC.

The NNTSC export protocol is complete now and happily exports live data to any clients that have subscribed to data streams that are being collected. Using this, I've been able to get the anomaly detection tool chain working with our SmokePing data right up to the eventing phase. Fixed a minor bug in the eventing code that would result in badly-formed event groups if the events do not strictly arrive in chronological order (which can happen if you are working with multiple streams of historical data).

Fixed a few libtrace bugs this week - the main one being trace_event being broken for int: inputs. It was just a matter of the callback function being registered inside the wrong #ifdef block but took a little while to track down.

Spent the latter part of my week tidying up my libtrace slides in preparation for a week of teaching 513 later this month.

This week I began my Masters under the supervision of Dr. Dean Armstrong at Virscient Limited and Dr. Richard Nelson. The purpose of my thesis is to develop a framework of simulation for IEEE 802.11ad using ns-3. 802.11ad is a new wireless standard for multi-Gbps throughput in the 60GHz spectrum. I will be extending ns-3 to support the new MAC and model physical and logical characteristics as well as designing a set of use cases for testing and showcasing the standard.

I have so far been reviewing material on ns-3 as well as learning a bit of Python and C++.

Shane suggested sending the protocol names once only to reduce the amount of redundant data sent each time and also, save on fifo space and bandwidth requirements. I designed a new protocol subheader for exporting protocol details(id, name, name_len) and these are sent to a client as soon as it connects to the server. Then, I had to chage the old exporting code and get rid of parts adding the name and name length and add in the appropriate code for the protocol IDs.

Then, I started working on exporting expired flow records to clients every X seconds(where X = 3mins/value chosen by user). I created a subheader for expired protocol records, and a structure for an expired flow record. Each time a flow expired, it was sent to be exported and its data added to the appropriate buffer. The buffer was then written to the FIFO when it filled up.

After I made sure that expired flow records were being exported correctly, I setup a timer which would export these records every X seconds, regardless of whether it was full or not.

Also got my Background chapter back from Shane. and started making the proposed changes.

The address data files for the planet lab runs were repaired and regenerated.

Cron scripts were written to automatically control and restart scamper data collection and a new first run was initiated. In addition more nodes were set up on planetlab as a safety margin, as various problems occur with nodes beyond my control, and this will ensure that data from 15 nodes is available.

Extra time for the planetlab slice has been requested and is now almost enough for the two month run time.

I have had an initial look at Tony's Internet Simulator 0. The code compiles OK on voodoo, however network data needs to be set up for an analysis to work. Other modes of analysis will need to be designed once I get a better understanding of how it works.