User login

Blogs

11

May

2015

I have been monitoring the latest black hole detector run on PlanetLab.

More changes based on Richards criticisms, to my thesis have been made. In particular I have added structure to the background chapter. This included finding some references relevant to the search for non 5-tuple fields that cause load balancing to occur. I found one paper that said they would look into this area in the future, thus helping to justify the research area.

11

May

2015

Continued keeping an eye on BTM until Brendon got back on Thursday. Briefed Brendon on all the problems we had noticed and what we thought was required to fix them.

Finished up my event detection webapp. Started experimenting with automating the running event detectors with a range of different parameter options. The first detector I'm looking at (Plateau) has about 15,000 different parameter combinations that I would like to try, so I'm going to have to be pretty smart about recognising events as being the same across different runs.

Started adding worker threads to anomaly_ts so that we can be more parallel. Each stream will be hashed to a consistent worker thread so that measurements will always be evaluated in order, but I still have to consider the impact of the resulting events not being in strict chronological order across all streams.

10

May

2015

Researched into document machine learning algorithms for processing small documents. Since social media is becoming ever popular there is a lot of work going into how to learn useful things from posts on social media, for example learning things about an event being posted on Twitter. I was especially interested in what techniques were used for Twitter since a message cannot be more than 140 characters, is posted by a user and has an interesting makeup of #hashtags etc. When we look at a log entry it is sort of similar in structure i.e. each event is a short line, is made by one specific program and has port numbers and IP addresses.

Some of the studies found better results by aggregating the "tweets" into one document which is already done with log files, others try and address the short nature of log files by looking at biterm pairs and many other techniques. Mostly the algorithms are based off topic modeling which infers topic's (groups of words) that could possibly generate the document, though I did find other clustering algorithms like spherical K-Means which I will look into further.

Looked into Mallet's API further and looked into how the importer works that creates the .mallet file that gets passed as input. So I was able to change the regex to parse tokens to include IP adresses and other numbers etc. and found it coverts the token strings to integers for storage. Then after getting the IP addresses etc. included in the input I tried it with topic modeling but it failed and the output was all weird characters so I need to find out what effect the numeric and punctuation characters have on both the input generation and modeling steps.

08

May

2015

Started making progress on the actual code behind the controller. So far I have it running, getting packets in and reading said packets. Ran into a bit of delay in figuring out how I'll create the rules since I started off by trying to create rules for DHCP communication, but since I don't know what ports on datapaths it's connected to to begin with, I need to learn them once I've started.

My initial thoughts are that I know the MAC/IP of my infrastructure that connects to the core switch, so I can create ARP packets, send them out the flood port and wait til I get a reply and handle them from there. Once I've done this, I can create the flows to allow DHCP traffic to get where it needs to go.

The next part is how does my controller know when a client is allowed to have internet access (Authorisation of AAA). I assume that one a client tries to talk to the WAN router, that information of where the WAN router lies has been given to them, so should be allowed to have access to it. The only problem here is that one could just know the configuration, set themselves up manually and go for it. I think this situation is okay for now, since this is less of a priority of getting things working in the first place.

06

May

2015

Rebuilt the bluetooth-next kernel with clean install of raspbian, still has input lag problems and breaks static IP setup (could be the keyboard).

Found another branch working with 802.15.4 implementation (linux-wpan-next) but negligible difference to the bluetooth-next branch, swapping to linux-wpan-next for kernel building as the openlabs blog quotes it as the one to use.

Source code has SoftMAC implemented with only sending a receiving implemented (basic functionality).
Nothing of HardMAC is implemented yet and no drivers present.

Working on setting up 2 rpi with same pan_id and channel then trying to exchange packets manually.

04

May

2015

This week and last I implemented a few basic filters, configured the time series graph scales and added a table displaying the most prominent flows and the associated device.
Turns out pcap doesn't store direction information so I couldn't implement the ingress/egress filter. pcap-ng supports direction though Libtrace doesn't support this format yet. Should be easy enough to implement when I have the information available.
Recently I have been planning my presentation which is this Wednesday.
Plan on seeing Brad either this week or next week to get NetFlow configured so I can start working with real data.

04

May

2015

Now that the per destination run is complete I started another run of black hole detector on Planetlab.

In response to the criticisms and suggestions from Richard I have been updating the background and related work chapters of my thesis.

In addition the IS0 runs, per destination and Megatree runs have resulted in updated graphs for my thesis. It is also means that I can write more in the results and discussion sections for these chapters.

04

May

2015

I spent last week researching and reading papers in the field of SDN. I've been writing summaries of the papers I'm reading.

There seems to be some research around how to best write SDN applications which interests me. In particular there is a notion that OpenFlow is to SDN what assembly is to software programming.

Even with my brief experience so far with my in progress fastpath addition to RouteFlow I felt it would be nice to have someway to add this as a modular addition - without extensive modifications to RouteFlow. Particularly given that essentially any OpenFlow application could benefit from fastpath, it would be nice not to re-write this into every application.

Their are some solutions to help solve this currently one such is frenetic, a language, which allows for serial and parallel combinations of network rules. This could be used to link functions together first apply firewall then route etc. Other abstractions such as network topology virtualisation allowing multiple physical switches to be treated as one single OpenFlow switch. This allows for much higher level control, focusing on only what the network should do.

CoVisor achieves something similar to frenetic, however is a hypervisor rather than a language. CoVisor provides an OpenFlow interface allow multiple controllers to connect and be combined together in a similar manner to frenetic, in parallel, serial or overridden. The idea being you deploy the best of any controller application on a network no matter what language it is written in.

04

May

2015

Continued keeping an eye on the BTM monitors. Changed several connections to use the ISPs DNS server rather than relying on the modem to provide DNS, which seems to have resolved many of our DNS issues.

Spent a bit of time digging into the problem of intermittent latency results for Akamai sites. It appears that our latency tests are interfering with one another as moving one of the previously failing tests to a new offset away from the others fixed the problem for that test.

Continued working on my Event Detection webapp. Added two new modes: one where the user does the tutorial, then rates 20 pre-chosen events and one where the user rates the same events without doing the tutorial. This will hopefully give us some feedback on how useful the tutorial is and whether the time required to complete the tutorial is worth it. Also added proper user tracking, with the generation of a unique code at the end of the 'survey' that the user can enter into the Mechanical Turk to indicate they have completed the task.

03

May

2015

Dean lent me a Segger JLink which I was able to use to start tracing down the hardfault. I was unable to do this, as the newlib library doesn't contain debug symbols.

While I could build my own, I created a work around exploting the fact that vsnprintf works fine.

The JLink is still very useful as I'm very likely to need to debug other issues later on. The XDS100v3 is pretty useless for this task. The JLink also provides a way better (faster, CLI) way to program the device so I suspect I'll even become more productive.

The next step is to try and get some reliable radio sniffing going. I need to ensure I can see 802.15.4 frames so that when I'm developing my own MAC I can make sure the frames are correctly formatted. This should be a trivial task as the CC2538DK comes with a USB dongle which there is already sniffing software for. I'm just unsure if I'll be able to do the sniffing under Linux.

tl;dr: I have UART, printf and malloc working. Now I need to start radio dev.