Database Block Diagram

Figure 1: Hardware Information Updates

Figure 2: Basic Database Configuration

Dataflow

Initial configuration

As highlighted in the server configuration, the Watchdog will observe a specified directory for incoming data in specific formats. Figure 1 highlights the information that the Watchdog needs to receive before any other data arrives. In order to relate incoming data, status information, and statistics to specific hardware, the hardware table needs to be populated. For that purpose, every sensor will generate a YYYY-MM-DD-hardware.json file at every bootup and drop it into the sync folder, sending it to the storage server. The watchdog will read this hardware file and populate the storage, hardware, RPi, SDR, and WR-LEN tables. With this data added to the database, it is ready to consume data.

Populating database tables

Every time a RF sensor records data, it will generate 3 files:

• A data file with extension .sc16 (should be changed to .fc32 if floats are used instead of integers).
• A metadata file with the same filename and a .json extension. This file contains information crucial for being able to process and use the data file.
• A status file ending in -status.json containing dynamic information such as device temperatures, remaining storage, CPU usage, etc.

As before, the Raspberry Pi will write these files to the sync directory and have them sent to the server for storage. On the server, the watchdog will move the files into the appropriate directory or add the status information to the database and then delete the status information file. Figure 2 illustrates how the Watchdog decides what to do with the 3 different options:

1. If it is status information file, add a row to the status table and update storage, hardware, RPi, SDR, and WR-LEN tables if necessary.
2. If it is a metadata file, add a row to the metadata table if this combination hasn’t been used yet.
3. If it is a data file, move it to the appropriate directory and add a row to the recordings table.

At the same time, a separate process watches the directory where data files are moved to and if it finds one that exists in the table, but doesn’t have any statistics in the outputs table yet, it will calculate the statistics, add them to the outputs table, and delete the file.

Database Structure

status_codes

Catalogs all possible status codes.