User login

Weekly Report -- 22/02/2013

25

Feb

2013

The new NNTSC now supports the LPI collector and installs nicely. Still waiting on Brendon to get his AMP message decoding code finished to his satisfaction and that will be the next thing to get working on the data collection side.

Also started developing a new data query / export mechanism that allows clients to connect and register their interest in particular data streams to receive ongoing live data as it is collected by NNTSC. The old approach for this involved the client explicitly stating the ID numbers for the streams they wanted data for, which was pretty suboptimal because it required knowledge that should really be internal to the database.

The other problem is that now we have different data table layouts for each different type of data stream, we need to inform clients about that structure and how to interpret the data stream.

All of this meant that I've had to design and implement an entirely new protocol for NNTSC data export. It's a request/response based protocol - the client can request the list of collections (i.e. the different data stream types), details about a specific collection (i.e. how the stream and data tables are laid out) and the list of streams associated with a given collection. It can then subscribe to a given stream, giving a start and end time for the time period required. If the time period includes historical data, that is immediately extracted from the database and sent. If the time period includes future data, the client and stream id is remembered so that new data can be sent to the client as it rolls into NNTSC.

Currently at the stage where we're getting historical data out of the database and preparing to send it to the client, with live data being the next main task to do.