User login

Brendon Jones's blog

08

Feb

2017

Updated packages for one of our AMP web servers, and spent some time updating configuration files to make use of new options that are available. Some of the options weren't being applied to all locations where they were relevant, so added those that were missing. Also fixed up errors when accessing various measurement data that didn't show up when testing with our sample data. Spent some time investigating performance issues that appear to have arisen with the new versions.

Started merging pull requests for amplet2-client that we got during NZNOG. Found and fixed a few other issues related to these where similar problems were occurring elsewhere.

Started doing background reading on a new project that we are about to start.

02

Feb

2017

Spent the first part of the week putting the finishing touches to the AMP tutorial that Shane and I presented at NZNOG on Wednesday afternoon. Put in a lot of time making sure that the instructions we were providing were clear, that our hardware set up would work correctly, and that we had covered everything we needed to.

Spent the second part of the week in Tauranga at the NZNOG conference. Got really good feedback and enthusiasm from the AMP tutorial, and the speakers for the Thursday and Friday were rather interesting.

16

Jan

2017

Spent most of the week working on the AMP tutorial for NZNOG. Installed the base operating system on our example hardware and documented the instructions that users will need to follow to get a working AMP client install. Also installed a base VM image and documented the instructions for them to get a working server install as well. Found and fixed a couple of small bugs in the web interface while creating sample screenshots of how to add tests. Generated some sample test output for each of the tests and wrote up a few points of interest about each one.

Built new packages for Ubuntu Xenial based on the work done at the end of last year and the bugfixes from this week. Currently trying to decide which (if any) final configuration steps are best performed during package installation, or if they should be left to the user to perform manually.

15

Dec

2016

Spent most of the week fighting with WSGI to get URI components containing slashes to properly pass through the routing and arrive at my code. Double escaping them will hide them enough from WSGI that the slashes aren't interpreted as a separator so that I can get a correct site name. Updated all the views, javascript and templates where a site/mesh name is used to be properly escaped.

Started adding a rudimentary email alerting system to netevmon to send emails when event groups cross a size/importance threshold. It's been a while since I looked at this code, so it's been a bit of a learning process to find the best place to do so.

07

Dec

2016

Wrote a short program to read from a RabbitMQ queue and dump the data as a CSV file. It had been requested for a particular site to get access to the data in other tools, and I think it's a helpful example for others wanting to do things with the data outside of what our tools provide.

Finished updating the site configuration web interface to show basic information about whether a site has a certificate or not (if using the amppki package), and allowing signing of the certificate in simple cases. Created a new user to run this web code. Found and fixed a few amppki user interface bugs as well, such as expired certs counting as duplicates (and triggering the simple protection code) when trying to revoke them.

Built and deployed some new packages for ampy and ampweb. Spent way too long trying to fix some issues caused by versioning of python modules where the Debian version is too old. Quietened a few unnecessary messages during package install.

28

Nov

2016

Finished and merged the scheduling REST interface after spending some time making return codes more consistent and implementing the last few missing functions. I can still see some of this changing to fit in with the way the web interface works, but it's done for now. Also fixed some minor bugs in ampweb where some sites weren't available in dropdowns.

Started adding the ability to display certificate status for each site in the web interface, and perform some very simple management there without having to use the command line tools. Split out some of the code from the command line tools into a library to be shared between them and the web interface.

Updated the ampy packaging scripts to bring the database schema up to date with the new changes when the package is updated. Also built new test packages for amppki to allow testing the certificate additions to the web interface.

Spent some time working with Jayden to help get him up to speed with AMP, and how one might go about writing a new test.

23

Nov

2016

Updated many of the pyramid routes to more accurately reflect the URLs that they are expecting and to make sure that only correct URLs containing all required parameters are passed to each route. Also fixed up many instances of generating route URLs used in href targets to properly set all the attributes.

Continued tidying up the new REST interface. Updated various scheduling/site/mesh queries to be better behaved when presented with odd data (distinguish between errors and lack of data), idempotent where possible. Added the ability to hide meshes from the matrix, so that various internal meshes used for scheduling etc don't show up, only those that you want to expose. Started writing some basic documentation about what endpoints are available and what they do.

Spent some time writing up ideas for a possible NZNOG AMP tutorial.

18

Nov

2016

Half the week was spent finishing up the packaging of the server components, and then installing them on amp.wand.net.nz. Still had some annoying database permissions issues that show up on fresh installs, but slowly eliminating them as I find them.

Chased down and fixed a few bugs that had been reported, including some unsigned latency values that were wrapping to very large values, and throughput graphs breaking in places where division by zero was possible due to unchecked values.

Spent some time installing and configuring event detection packages on another site. Ran into some issues where events weren't being generated and worked with Shane to try to track it down. The issues seemed to be related to having collections enabled but with no tests being performed - there was no data to fetch so it didn't realise that it was up to date.

Got a small amount of work done tidying up the scheduling web interface to be more easily exposed for others to use. Planned out the URL structure, updated most of the interface to use the new URLs and started to add a few new functions that could be useful.

07

Nov

2016

Spent the week working on updating packages for all the server-side components (ampweb, amy, nntsc, netevmon, etc). This was made harder than it should have been by me making the build dependencies list more accurate, causing a different build system to be used, which then tries to pull in further dependencies from PyPI (that are unavailable as packages in some Debian flavours) and build them on a machine that doesn't have (and shouldn't have) the libraries to do so. Ended up reverting some of the changes in the interests of getting it to work, and will revisit this another time.

Made various improvements to the packaging scripts, such as tidying up enabling events when ampweb and netevmon are both present, regardless of the order that they were installed, installing the new influxdb database, fixing permissions, and making sure users exist. Went through multiple test deployments on both Wheezy and Jessie to make sure that the upgrade process will work smoothly next week.

02

Nov

2016

Removed the separate amplet2-client-lite packages that ran without a local rabbitmq server and updated the regular packages to determine if rabbitmq is present or not. This makes the Debian packaging a lot easier and also means that there are fewer options that the user needs to worry about getting correct.

Started to work on making the URLs used to access the scheduling web UI a lot more consistent and sensible, and as RESTful as possible. I want to be able to expose this to users so they can programmatically modify the test schedule using outside tools, as well as making the internals a lot easier to maintain.