Thursday, January 11, 2018

Monitoring the Monitors: Removing NaNs from rtl_power data and signal spike dection (Part 2 ½)

My first attempt at using rtl_power to capture motion detector signals was a bust. The data was riddled with NaN values, making it useless. Stumped for options, I dropped an e-mail to the author of this rtl_power article. His answer couldn't have been simpler: are you sure you're using the most up to date version of rtl_power?

I wasn't. As I had noted, to install rtl_power power on my Raspberry Pi I followed this transcript. It pulls the source for rtl_power as follows:

$ git clone git://git.osmocom.org/rtl-sdr.git

The latest version of the code, which fixes the NaN issue can be found here: https://github.com/keenerd/rtl-sdr.

I re-ran the build commands as suggested in the transcript (substituting the right git source URL) and found myself with a new version of rtl_power. A quick test showed that my NaN's were now gone!

Here's some test data:

I recaptured a short snapshot of motion detection data using rtl_power. The first set of data captures signals when there's no motion; the second set of data contains signals generated when I was tripping the motion detector. I did this by running commands like the following:

for i in `seq 1 3` ; 
  do rtl_power -f 344M:346M:20K -e 30 -i 1 > nomotion.$i.csv ;
  sleep 10;
done

I then wrote a quick PHP script to detect spikes in the data. My hope was that the 'nomotion' files would contain no spikes, while the 'motion' files did have spikes. Here's that script:

<?php
/*
 * A PHP file for reading in a rtl_power CSV file and printing out any blips
 */
if(count($argv) != 2) {
  echo "Usage: php -f {$argv[0]} data.csv\n";
  exit;
}

$last_snap      = array();
$data_col_start = 6;
$blip_threshold = 6;
$fd             = fopen($argv[1], "r");


while($row = fgetcsv($fd)) {
  $row = array_map('trim', $row);

  for($col_i = $data_col_start; $col_i < count($row); $col_i++) {
    $bin  = $col_i - $data_col_start;
    $freq = number_format(($row[2] + ($row[4] * $bin)) / 1000000, 4);
    $level = $row[$col_i];

    if(isset($last_snap[$freq])) {
      $diff = abs($last_snap[$freq] - $level);
      if($diff > $blip_threshold) {
        echo "$freq:$bin:$diff:$level\n";
      }
    }

    $last_snap[$freq] = $level;
  }
}
?>

Notice the value above for blip_threshold. This is an arbitrary value and one that I arrived at through experimentation. My thinking is that there's naturally noise in the system, so even when the motion detectors aren't sending messages, there's going to be some spikes.

And here's the output of running the PHP code against some sample files:

$ php -f ../find_blips.php nomotion.1.csv  # whoo! No spikes detected!
$ php -f ../find_blips.php motion.1.csv
345.0000:0:10.96:-7.35
345.0200:1:7.33:-11.10
344.8800:44:6.36:-10.89
344.9000:45:7.75:-9.28
344.9200:46:10.18:-6.98
344.9400:47:15.02:-2.38
344.9600:48:25.33:7.70
344.9800:49:22.82:5.56
344.9000:45:6.41:-14.93
344.9200:46:7.46:-13.25
344.9400:47:8.54:-10.04
344.9600:48:9.28:-0.70
344.9800:49:9.12:-2.67
344.9400:47:7.37:-17.41
344.9600:48:16.83:-17.53
344.9800:49:14.81:-17.48
345.0000:0:11.24:-18.45
345.0200:1:7.57:-18.61
344.8800:44:7.13:-10.38
344.9000:45:9.06:-8.32
344.9200:46:11.63:-5.84
344.9400:47:15.94:-1.53
344.9600:48:26.24:8.60
344.9800:49:23.74:6.49
345.0000:0:11.17:-7.24
345.0200:1:7.07:-11.35
344.9000:45:6.95:-17.28
344.9200:46:9.41:-17.28
344.9400:47:13.88:-17.59
344.9600:48:23.95:-17.55
344.9800:49:21.77:-17.50
345.0000:0:12:-18.34
345.0200:1:8.32:-18.42
344.9000:45:6.64:-10.71
344.9200:46:9.24:-8.15
344.9400:47:13.52:-4.02
344.9600:48:23.69:6.02
344.9800:49:21.35:3.94
345.0000:0:11.34:-6.87
345.0200:1:7.91:-10.70
344.8800:44:7.33:-17.05
344.9000:45:8.15:-16.01
344.9200:46:10:-15.14
344.9400:47:11.76:-12.49
344.9600:48:13.38:-4.09
344.9800:49:12.94:-5.71
344.9600:48:13.49:-17.58
344.9800:49:11.8:-17.51
345.0000:0:11.18:-18.45
345.0200:1:7.78:-18.74
344.9200:46:6.52:-10.92
344.9400:47:10.16:-7.38
344.9600:48:20.14:2.51
344.9800:49:18.03:0.50
345.0000:0:14.07:-4.38
345.0200:1:10.15:-8.47
345.0400:2:7.42:-10.80

Will you look at that?! I'm detecting spikes in the motion data, and can filter out noise in the nomotion data. I've got to say, this is really promising.

Up next is to refine this and map spikes to specific monitor detector behavior.

A huge thanks keenerd for making all of this possible!

No comments:

Post a Comment