User login

Weekly Report -- 15/02/2013




The development of NNTSC took another dramatic turn this week. After conferring with Brendon, we realised that the current design of the data storage tables was not going to support the level of querying and analysis that he wanted for AMP data. This spurred me to quickly write up a prototype for a new NNTSC from scratch that allowed each different data collection method to specify exactly how the data table should look. This means that instead of having one unified data table with the inflexible schema of (stream id, timestamp, data value), we now have an AMP ICMP test data table that is (stream id, timestamp, pkt size, rtt, loss, error code, error type) and a Smokeping data table that is (stream_id, timestamp, uptime, loss, median, ping1, ... ping20).

We've also done away with the central queue and simply given each data parser its own connection to our database. This fixes a problem I was having where trying to read data from a file too fast was causing the queue to fill up and run the machine out of RAM.

Smokeping data collection is now working with the new NNTSC, so I now need to write the data parsing modules for each of the other input sources we used to support as well as re-do all the nice installation script stuff I had done for the previous version of NNTSC.