I think I've missed two of these...
I've spent a bunch of time re-wrorking my packet scheduler this past week. I wanted to allow for link-layer retransmission of packets which ht epacket scheduler was not originally designed to do. It compiles, and messages get sent however the upper MAC layers manage their own re-transmissions and ACK messages no longer get passed up and so nothing works just yet. This is a simple case of re-working the state machine to accept the new packet states; ACK, NO ACK and NOT SENT (in the case the packet fails CSMA and never leaves the radio).
ubiquiOS (which seems to be the topic of my last report) is well integrated and working. Before the packet scheduler re-work, I had 3 nodes able to simultaneously associate with the gateway node (pan coordinator). After association, the nodes would ping the gateway and measure the response times. With a beacon period of 500ms, the response time averaged at 500ms with 1% packet loss.
Adding link layer retransmission should improve that packet loss figure.
After that, I want to start working on AES. I'll hard code the keys for now, and try to install them at a sensible time. At which point, I can celebrate because that's the infrastructure I needed to start my project! I can then remove the hard coded keys, and replace them with authentication and key exchange.
Stretch goal is to add 6LoWPAN on top - which shouldn't be too difficult, as I've essentially made a pipe that correctly sized packets can be piped into.
REST API works a charm, freeradius can authenticate users and add OpenFlow rules to allow the customer to talk to a router.
VLANs are hard, it breaks everything and I'm yet to find a solution for this. Matching on 2 vlans is still not a thing and handling a single vlan, stripping it and throwing it at a second table doesn't work. It seems OVS can only process one VLAN per run through the switch, which sucks.
Talked to Chris at Lightwire about his solution, which is hack the code until it works. I'm probably not going to be able to get Lightwire IP out of him for the project, and it sounds hard. I'll have to come up with some other solution.
This week has focused on the check-paths routine for validating the IS0 simulator. I recently found that this routine was getting into an endless loop, though it did not exhibit runaway memory usage. The problem was that it was looking for a next hop and replying not found, and so on. It turned out that it was not reading missing hop identifiers from the hops-dest file of hop information, and so there were gaps in some traces when read into the hash tables. Another problem was that reading the TTL value from the log file looked for an equals sign that didn't exist, and so the whole line was read as the TTL value causing the hash table to be corrupted. Finally the simulator message string was too short to print some of the longer messages. This also resulted in some errors when analysing the trace log file, as for example some addresses were truncated and were flagged as incorrect by the program.
Short week as I was on leave on Thursday and Friday.
Continued tweaking the event groups produced by netevmon. My main focus has been on ensuring that the start time for a group lines up with the start time of the earliest event in the group. When this doesn't happen, it suggests that there is an incongruity in the logic for updating events and groups based on a new observed detection. Now the problem happens rarely -- which is good from the perspective that I am making progress but it is also bad because it takes a lot longer for a bad group to occur so testing and debugging is much slower.
Spent a bit of time rewriting Yindong's python trace analysis using C++ and libflowmanager. My program was able to run much faster and use a lot less memory, which should mean that wraith won't be hosed for months while Yindong waits for his analysis to run.
Added a new API function to libtrace to strip VLAN and MPLS headers from packets. This makes the packets easier to analyse with BPF filters as you don't need to construct complicated filters to deal with the possible presence of VLAN tags that you don't care about.
Installed libtrace on the Endace probe and managed to get it happily processing packets from a virtual DAG without too much difficulty.
I've done some basic benchmarks (for loops around operations) and made some optimisations based upon that for the libraries I've been testing. All the libraries performed well enough for my purposes. The performance of the libraries did differ depending upon the task being performed, with some outperforming the others at one task and the reverse with another task. Overall I've found ROFL to be the easiest to work with and it appears to still be in active development.
I've have now got access to the OFLOPS-Turbo source. So I will be looking at that to see what changes have been made and if it is more suitable to build on top of.
I've focused my attention back to RouteFlow again and started to fix some problems. I've investigated the issue with discovering hosts (i.e. ARP). Previously we were not sending IP traffic to the controller meaning that we did not already know about a host we would drop all traffic destined to it. I investigated using broadcast (ping, arp, dst mac broadcast etc.) methods to try find all hosts proactively however this did not work as most machines were not listening. As such I concluded we still have to send unknown IP traffic to the controller so it can perform ARP. This is a DOS vector, but should be less of a concern now with fastpath.
Manged to set up static IP for the gateway device:
required a clean install of the OS and radvd (had multiple version of radvd installed leading to unusual behavior).
Adding static config the /etc/network/interfaces file, then required an ifdown/ifup to refresh the configurations of the interface. Needed to add the interface the ifstate file in order for if* to work (/run/network/ifstate added: lowpan0=lowpan0) however this file is not persistent so changes may need to be replicated each boot.
Upon reboot the static config was missing - still looking into the source of the problem (address is assigned after a ifdown/ifup on the lowpan0 interface.)
Installed bind9 as the dns services, configured local zone and have working ipv4 local name resolution, still checks outside if local cannot resolve name (kept it this way in case the gateway needs to be connected to the internet), however it does not have working ipv6 name resolution yet.
Talked to brad about getting tagged log file data and he modified bearwall to log both packets that were allowed and packets that were blocked, these are indicated in the event in the log and in the case they were blocked the reason is indicated. So now I have a days worth of tagged firewall logs (about 11,000 lines).
I then made an application to process this data and convert it into the ARFF format so that it will be compatible to work with in WEKA. Now I need to split this into separate sets for testing, parameter tuning and evaluation.
Now I will build the application with the WEKA framework to rank this data based on classification accuracy.
Got a little bit of work done this week. Have setup freeradius3 in a new deb8 VM, have bridged device to mininet test environment and can hand out DHCP leases without too much hassle. Currently, there's no VLAN tags on this otherwise it breaks, so need to investigate freeradius sitting on multiple vlans (vlan per hop) handing out DHCP to double tagged traffic (triple if you count the HOP vlan).
Also added rest api support to the controller, will build on that once I have the VLAN task sorted.
Spent some more time looking into using embedded Chromium as part of the
HTTP test. I've managed to successfully extract all of the
navigation/resource timing information from the browser after the page
has loaded which is very useful. Getting access to headers looks like it
will require implementing my own resource handlers and processing
requests manually, but should be doable. Also, I still haven't managed
to completely decouple the browser from GTK - something is still trying
to initialise it even though there is no need for it and nothing is ever
drawn to the screen.
Tidied up some more configuration parsing and parts of the main loop in
the amplet client, removing the need for a few more global variables
that were convenient at the time of writing.
Helped Brad configure a new measurement machine to be sent out.
Reconfigured some of the existing machines to swap the management ports
around so we can test them without the reporting traffic interfering.
Tried multiple setups of CoAP servers:
libcoap has been most successful so far, the example provided functional server with time functionality and worked nicely with Copper (Cu) a Firefox extension for CoAP servers.
Can configure the address (-A) to start up with - one thing that wasn't included was the necessity to add '&' to the end in order for the command to execute (possibly something within the code)
Tested with a LAN machine (/w Firefox + Copper) and the IPv4 address of the RPi on LAN
Initial Copper screenFirst view of the CoAP server from inside Copper
Selecting 'time' and clicking 'observe' produces the following result
ObservingObserving the time feature of the CoAP Server (PUSH notifications)
Running the client example program with settings "echo 1000 | sudo ./coap-client -m put -T cafe coap://192.168.1.7/time -f -"
gave the output: v:1 t:0 tkl:4 c:3 id:18275
which is just the header of the CoAP message (no payload was defined)
Also trying to 'put' into the core or async (instead of time) gives a response of "Method Not Allowed" same as trying the operations in Copper.
Having a running example is useful and next steps will be to configure the example, or write a separate script based on the example with the functionality that is required for the projects server.
ccoap is a C based implementation of a CoAP server with examples, however running anything other than the examples (there are command line versions) fails to work (no response from the server. Also required editing of /etc/services and installing cmake to compile the code.
WebIOPi is an interesting setup, has functionality for both CoAP and HTTP servers which could come in handy later on in the project. The main feature of the project is to have a web interface for viewing the GPIO ports on the board. it also is implemented as a daemon service, which is part of what is wanted for the project server.