User login

Blogs

16

Jan

2017

Spent most of the week working on the AMP tutorial for NZNOG. Installed the base operating system on our example hardware and documented the instructions that users will need to follow to get a working AMP client install. Also installed a base VM image and documented the instructions for them to get a working server install as well. Found and fixed a couple of small bugs in the web interface while creating sample screenshots of how to add tests. Generated some sample test output for each of the tests and wrote up a few points of interest about each one.

Built new packages for Ubuntu Xenial based on the work done at the end of last year and the bugfixes from this week. Currently trying to decide which (if any) final configuration steps are best performed during package installation, or if they should be left to the user to perform manually.

16

Jan

2017

Spent a bit of time testing out some of Brendon's AMP tutorial instructions, making sure that everything so far is sane and no steps are missing. I anticipate there will be a lot more of this next week as the tutorial gets closer to a complete draft.

Continued working on verifying and fixing the auto-generated FSMs. Going over the entire set of generated FSMs from my test dataset threw up a number of bogus looking machines, so I've been working on investigating and (when necessary) fixing the problems. I've also managed to get self-repeating states working correctly for the most part; just one or two edge cases that still need to be detected and handled properly. Re-implemented tagging the original call logs with the FSMs that were matched by subsequences within the call log -- the current implementation is naive in that it assumes any state within a machine could be a start state, which is not going to scale well so I need to come up with a way to infer potential start states (or at least rule out definite non-start states).

Re-worked libflowmanager to be usable in a parallel situation. Previously, the flow map was a global variable. Now, you can have multiple flow maps so you can have one per thread and use libtrace's bidirectional hashing to ensure that each flow corresponds to only one thread, and therefore only one flow map.

Started experimenting with using parallel libtrace with libprotoident applications. I soon ran into a bug where using the built-in hasher thread to distribute packets could cause a deadlock, so spent most of Friday trying to track this down.

13

Jan

2017

This week I worked on my sack amp test, which requires a basic implementation of TCP to handshake and send a HTTP request so it has data to then send a selective acknowledgement. Got it to the stage where it can handle expected behavior.

I also spent time giving my sack test the ability to craft packets with data and with a larger range of options. This was done to be able to send HTTP requests and the sack option.

09

Jan

2017

Back to work for two days this week. Caught up on a pile of email, then wrote my talk for NZNOG later this month.

Tested and released a new version of libprotoident.

Started working on adding single node loops to my FSMs for the STRATUS project.

06

Jan

2017

Libprotoident 2.0.10 has been released.

This release includes rules to match new traffic patterns for many of the protocols that we introduced in the 2.0.9 release. We've also added two new protocols: BACnet and Maxicloud.

This release also no longer treats TCP keepalive packets as payload-bearing.

The full list of updated protocols can be found in the new libprotoident ChangeLog.

Download libprotoident 2.0.10 here!

19

Dec

2016

Tidied up and documented the FSM extraction code, so that I'll be able to remember how it works when I start working on it again in earnest next year.

Finished the matrix layout / selection changes and merged them back into develop. Hopefully we will get a chance to roll these out early next year once Brendon builds some new packages.

I had to run a test capture for a few days last week to make sure that some changes Richard had made to libtrace had not broken DAG and RT inputs. Ran the resulting traces through libprotoident to see if there are any new protocols worth investigating. Managed to make a few improvements to the rules for existing protocols to catch a few cases that we were missing but otherwise nothing particularly exciting cropped up.

16

Dec

2016

This week I tested my second AMP test and ironed out the remaining issues and bugs. Because all the options where set in my TCP ack packet, if it was dropped due to one of the options I wouldn’t know why so I added a fall back which would test options individually if a few preconditions where meet. I also separated my tests into their own files to make it easier to see what was going on and make extensions easier as well as a generally refactoring of the code.

15

Dec

2016

Spent most of the week fighting with WSGI to get URI components containing slashes to properly pass through the routing and arrive at my code. Double escaping them will hide them enough from WSGI that the slashes aren't interpreted as a separator so that I can get a correct site name. Updated all the views, javascript and templates where a site/mesh name is used to be properly escaped.

Started adding a rudimentary email alerting system to netevmon to send emails when event groups cross a size/importance threshold. It's been a while since I looked at this code, so it's been a bit of a learning process to find the best place to do so.

12

Dec

2016

In Wellington for STRATUS forum on Monday. Had a few interesting chats -- definitely a lot of people out there interested in anomaly detection in a variety of contexts.

Continued refining my FSM generation code. Managed to get rid of most of the obviously incorrect transitions in my test cases now. There's still a bit of work to do in terms of tidying up some orphaned states that are left over as a result of the code realising they are redundant and trying to choose better start states, but my main focus before the end of the year will be tidying up the code and making sure it is sufficiently documented so I'll be able to pick it up again in the new year.

Fixed a bunch of small problems with amp-web and NNTSC that we've known about for a while. Started working on replacing the matrix selection tabs with dropdowns and combining related "tabs" into a single matrix type, e.g. http duration and http page size are combined into a single "http" matrix with the ability to change the metric using a dropdown.

09

Dec

2016

This week I worked towards getting an amp test complete that will test a list of TCP options to a location, seeing which options are accepted or if the packet is dropped all together. With the idea being that if this was done from many points in the network you could infer if a middle box is tampering with the options.