Tuesday, June 30, 2020

A Programmer's Understanding Of e

Imagine I get a request from a bank to program the function that calculates money earned from one of their accounts. This seems too easy, but here's the code:

(define ($ amt)
  (inexact amt))


(define (grow/1 balance rate)
  (let ((interest (* balance (/ rate 100))))
    ($ (+ balance interest))))

;; Starting balance is $37 and growing by 8%
> (grow/1 37 8)
39.96

;; Confirm an edge case. 0% means no growth.
> (grow/1 37 0)
37.0

;; Another edge case. Growing 100% means doubling your money.
> (grow/1 37 100)
74.0
> 

"Close," the bank replies, "but not quite." The issue: we need to support growing the money in multiple increments. My imaginary contact explains:

Along with promising our customers 8% interest, we also tell them that their money will grow quarterly. So rather than one 8% increase at the end of the year, we give four 2% increases throughout the year.

That's not too tricky to program. Now my growth function takes in an additional frequency argument:

(define (grow/2 balance rate frequency)
  (let ((r (if (or (= 0 rate) (= 0 frequency))  0
               (/ (/ rate frequency) 100))))
    (let loop ((balance balance) 
               (frequency frequency))
      (cond ((= frequency 0) ($ balance))
            (else
             (let* ((interest (* balance r)))
               (loop ($ (+ balance interest))
                     (- frequency 1))))))))
;; The same $37 at 8%, but now grown quarterly
(grow/2 37 8 4)
40.04998992000001

;; The 0% interest edge case still produces no growth. Good.
> (grow/2 37 0 4)
37.0

;; Growing by 100% now *more* than doubles your money. Nice!
> (grow/2 37 100 4)
90.33203125

Growing the money 4 times a year at 8% gives a slightly better return. But check out that 100% edge case: that now more than doubles the money in the account. Looking at these examples one wonders if we can get a better return just by having the bank grow the money more often. Let's try it:

;; Quarterly - good.
> (grow/2 37 100 4)
90.33203125

;; Monthly - better.
> (grow/2 37 100 12)
96.68230573831309

;; Daily - best!
> (grow/2 37 100 365)
100.43899683480936

;; 1000 times per year
> (grow/2 37 100 1000)
100.52618549272809

;; 5000 times per year
> (grow/2 37 100 5000)
100.56637185376789

This strategy of incrementing the balance more often starts off promising. If we can get the bank to increment our money every day for a year we'll make a little over $100, versus the $90 we'll make growing the money quarterly. However, there's a limit to this strategy. Growing the money 1000 or 5000 times doesn't get us past the $100 return.

This makes sense: we may be able to convince the bank to grow our money 5000 times a year instead of 4--which sounds good--but each increase is using an itty bitty (a technical math term) interest rate.

So this is intereting, but what does this have to do with the math constant e.

Check this out: if you look up e you'll see that its value is approximiately:

  e = 2.718281828459045

Now let's say we run our growth/2 function on the simplest possible arguments: $1 grown at 100%. We get:

> (grow/2 1 100 100000)
2.718268237174493

That number is pretty close to e. Coincidence? Nope. An Intuitive Guide To Exponential Functions & e explains:

The number e (2.718…) is the maximum possible result when compounding 100% growth for one time period. Sure, you started out expecting to grow from 1 to 2 (that’s a 100% increase, right?). But with each tiny step forward you create a little dividend that starts growing on its own. When all is said and done, you end up with e (2.718…) at the end of 1 time period, not 2. e is the maximum, what happens when we compound 100% as much as possible.

So e isn't some mysterious number. It's the result of (grow/2 1 100 x) where x is very large number. So large that it's just about infinity. As we suspected, e is a limit:

e is like a speed limit (like c, the speed of light) saying how fast you can possibly grow using a continuous process. You might not always reach the speed limit, but it’s a reference point: you can write every rate of growth in terms of this universal constant.

While it's novel that we can discover this limit ourselves, that's not the whole story.

Back in 2018 I experimented with an animation framework with an odd quirk. You could draw exactly one thing: lines from (0,0) to (0,1). You could then scale, rotate and translate these lines. So while you could only draw one type of line, you could transform their size, shape and position in whatever you imagined. You can play a similar game with e. By raising e to a power, you can scale it to calculate any growth rate. Including negative growth (I'm looking at you radioactive half-life).

As a sign of how frequently one raises e to various powers Scheme names this function simply exp. Here's yet another grow function, this time powered by e.

(define (grow/3 balance rate)
  ($ (* balance (exp (/ rate 100)))))

> (grow/3 37 8)
40.08162150397347
> (grow/3 37 0)
37.0
> (grow/3 37 100)
100.57642765298466

So not only do we have a sense of what e is, we now have a new tool in our tool kit: ex. If you find yourself dealing with problems relating to continuous exponential growth, like say an out of control pandemic, e is your friend.

Thursday, June 25, 2020

But the Car Can't Do That | Lessons From a Mercedes A220 UI Glitch

Shira starts up the car and drives away. A few minutes into her drive she looks down to see this:

That's a Mercedes A220 dashboard minus all the gauges. She wasn't stopped for speeding, but had she been the answer would have been: no officer, I have no idea how fast I was going. That's because she had no speedometer on her display.

The A220's fully digital dashboard is sweet. You can switch up themes to change the look and feel. But alas, because it's software, it can also be buggy. And apparently Shira tripped over just such a bug. On 495. Going 60'ish miles per hour.

At a stop light she cut the engine and started up the vehicle again. There was no change. After leaving the car parked for a few hours, it went back to normal.

I recently had a similar, though far less dramatic experience. My ASUS C302 Chromebook wouldn't charge. There were only three explanations I could imagine: the outlet was bad, the cable was bad or the computer was bad. I tried another outlet and found it wasn't at fault. I ordered a new cable from Amazon and found it wasn't at fault. That meant that the charging port on the computer was busted. Or was it?

The first hit of a Google Search turned up an alternative explanation:

Hello all, I got an Asus chromebook flip about a month ago and today I plugged it in to charge and I got no response from the computer or the little indicator on the side.
...
This happened to me even though I was using the stable channel. I found another post on here about holding Refresh while hitting the Power button to perform an EC Reset (Reset charging controller).

Sure enough, holding down refresh and the hitting the power button fixed the problem. Apparently the software handles charging the device ran into an issue.

So is software an awful replacement for hardware and we're all doomed? Uh, no. As a programmer I'm hardly ready to give up on the terrific power and flexibility that software introduces into a system. But also as a programmer, I think it's important to stay humble.

When topics like electronic voting come up it's worth remembering examples like the above. "It'll be foolproof" they tell us. At which point I'll kindly chime in with "let me tell you about the time my wife found herself speeding down the highway without a clue as to her speed."

It boils down to this: Software is awesome. Proceed with caution.

Friday, June 19, 2020

An Ultra-Lightweight PHP Unit Test Framework

A project of mine called for a major DB re-org. The result was going to correct a number of longstanding system issues. It was also going to break the entire application. In theory, once the library code that powers the app was updated to work with the new database structure, the software would be fixed. To insure I addressed all the issues I decided to write a suite of unit tests and watch as they slowly went from all failing to all succeeding.

Given that the application is written in PHP, the obvious choice for a unit test framework was PHPUnit. After reading the docs, however, I couldn't resist putting together my own framework. It had three guiding principles.

First, I wanted tests to be easy to write. The convention I arrived was to have a collection of files nested under a top level tests directory. Each file would contain a variable named $tests that was an array of functions. Each function represents a single test. If the function executes without throwing an exception it is considered passing. That description sounds complex; in pracice the code is pithy:

 $ cd tests/utils/date-and-time
 $ cat time.php
 <?php
   $tests = [
     function() {
       $t1 = strtotime("tomorrow 3:00pm");
       $t2 = strottime("+1 day 3:00pm");
       assert($t1 == $t2);
     },
     
     function() {
       $t1 = time() + (60*60*24*3) + (60*60*3);
       $t2 = strototime("+3 days 3am");
       assert($t1 == $t2);
     }
   ];
 ?>

assert is a standard php function and can be configured to throw an exception. Though any code that throws an exception can be used to fail a test. By catching exceptions, it's possible to confirm that negative paths are working too. For example:

  $tests = [
    function() {
      $name = gen_unique_username();
      $u1 = new_user(['username' => $name]);
      try {
        $u2 = new_user(['username' => $name]);
        assert(false, "Uh, should have failed with duplicate user ex");
      } catch(DuplicateUserException $ex) {
        assert(true, "Hurray, we got a dup user ex");
      }
    }
  ];

Beyond picking the path for the .php file, nothing else is named. That's by design, as it makes composing tests that much simpler.

The next principle I was after was to make the test suite easy to run. Because the framework is so lightweight, it's straightforward to create top level programs that run the tests. For example, here's a test driver that runs from the command line:

<?php
require_once(__DIR__ . '/../lib/siteconfig.php');
(php_sapi_name() == 'cli') || die("I'm a command line tool, thanks.");


function show_outcome($file, $index, $error = false) {
  $path = preg_replace('|^.*?/tests/|', '', $file);
  echo "$path:$index:" . ($error ? $error : "pass") . "\n";
}
$stats = testing_run_all(['fail' => 'show_outcome']);

  
echo "\nPass: {$stats['pass']}, Fail: {$stats['fail']}\n";

And here's a similar version that runs from within the web app itself:

<?php
require_once(__DIR__ . '/../lib/siteconfig.php');
header("Content-Type: text/plain");
(g($_GET,'key') == 'a8a49399f8073e7f26d87a12771b954f') || die("Access Denied");

function show_outcome($file, $index, $error = false) {
  $path = preg_replace('|^.*?/tests/|', '', $file);
  echo "$path:$index:" . ($error ? $error : "pass") . "\n";
}
$stats = testing_run_all(['fail' => 'show_outcome']);

  
echo "\nPass: {$stats['pass']}, Fail: {$stats['fail']}\n";

And here's a version that's intended to run from cron and doesn't output the details of the tests. Instead, it logs the outcomes to AWS's CloudWatch, which make the data trivial to include in a system-wide dashboard.

<?php
require_once(__DIR__ . '/../lib/siteconfig.php');

(php_sapi_name() == 'cli') || die("I'm a command line tool, thanks.");

function show_outcome($file, $index, $error = false) {
  $cloudwatch = get_cloudwatch_instance();
  $cloudwatch->putMetricData([
    'Namespace' => 'app-unit-tests',
    'MetricData' => [
       [ 'MetricName' => $error ? "fail" : "pass",
         'Timestamp'  => time(),
         'Value'      => 1,
         'Unit'       => 'Count' ]
     ]
  ]);
}
testing_run_all(['fail' => 'show_outcome']);

Finally, I wanted make the framework easy to embed into other projects. By keeping the framework lean, I was able to hit this goal. Recently I got to test this out by packaging unit tests with a custom WordPress plugin. The tests directory was out of the way, and a WordPress friendly test running program was easy to write. Here's the test runner's code. Note how I'm using the prefix 'foo', it assumes I'm authoring with 'foo' plugin.

<?p<?php
/*
 * A PHP file for implementing a trivial unit test driver
 */
require_once(__DIR__ . "/../../../../wp-config.php");
header("Cache-Control: no-cache");
header("Expires: 0");
(foo_g($_GET, 'key') == '81b05f7ad603f5d7ae925e44d08ae14b') || wp_die("Permission Denied");
?>
<html>
  <head>
    <style>
     table {
       border-collapse: collapse;
       border: 1px solid #222;
       width: 100%;
     }
     table td {
       border-top: 1px solid #666;
       vertical-align: top;
     }
     td, th {
       padding: 1em;
     }
    </style>

  </head>
  <body>
    <table>
      <tr>
        <th>Test</th>
        <th>Outcome</th>
      </tr>
      <? $stats = foo_testing_run_all(['fail' => 'show_outcome', 'pass' => 'show_outcome']); ?>
    </table>

    <h2>Outcomes</h2>
    <p>
      <b>Pass:</b> <?= $stats['pass'] ?>
    </p>

    <p>
      <b>Fail:</b> <?= $stats['fail'] ?>
    </p>

  </body>
</html>

<?
function show_outcome($file, $index, $error = false) {
  $path = preg_replace('|^.*?/tests/|', '', $file);
  echo "<tr>";
  echo "<td>$path:$index</td>";
  echo "<td><pre>" . ($error ? $error : "pass") . "</pre></td>";
  echo "</tr>";
}

Below is the code for the test framework. Feel free to grab and use it. Remember: tests get easier and more addictive the more you write. Like, say, version control once you've setup your environment and overcome the learning curve you'll wonder how you ever lived without this tool.

<?php
// shared/lib/testing.php -- test framework
function testing_run_all($options = []) {
  return directory_deep_fold(__DIR__ . '/../../tests',
                      function($file, $carry) use($options) {
                        $stats = testing_run($file, $options);
                        return [
                          'pass' => $stats['pass'] + $carry['pass'],
                          'fail' => $stats['fail'] + $carry['fail'],
                        ];
                      }, ['pass' => 0, 'fail' => 0]);
}

function testing_run($file, $options) {
  set_error_handler('exception_error_handler');
  ini_set('zend.assertions', true);
  ini_set('assert.exception', true);

  $pass_handler = g($options, 'pass', function($file, $index) {});
  $fail_handler = g($options, 'fail', function($file, $index, $cause) {});
  require_once($file);
  $stats = ['pass' => 0, 'fail' => 0];
  $ctx = [];
  if(isset($tests)) {
    foreach($tests as $i => $t) {
      try {
        $ctx = $t($ctx);
        $pass_handler($file, $i);
        $stats['pass']++;
      } catch(Throwable $ex) {
        $stats['fail']++;
        $fail_handler($file, $i, $ex);
      } catch(Exception $ex) {
        $stats['fail']++;
        $fail_handler($file, $i, $ex);
      }
    }
  } else {
    $fail_handler($file, 0, new Exception("No variables \$tests defined"));
    $stats['fail']++;
  }
  return $stats;
}

// Utility Functions
function g($array, $key, $default = false) {
  return array_key_exists($key, $array) ? $array[$key] : $default;
}

function directory_deep_fold($root, $fn, $carry) {
  $dh = opendir($root);
  while($file = readdir($dh)) {
    if($file == '.' || $file == '..') {
      continue;
    } else if(is_dir("$root/$file")) {
      $carry = directory_deep_fold("$root/$file", $fn, $carry);
    } else {
      $carry = call_func($fn, "$root/$file", $carry);
    }
  }
  return $carry;
}

// Borrowed from: https://www.php.net/errorexception
function exception_error_handler($severity, $message, $file, $line) {
  if (!(error_reporting() & $severity)) {
    return;
  }
  throw new ErrorException($message, 0, $severity, $file, $line);
}

Thursday, June 11, 2020

The Magic of Plant Snap and The Taste of Honeysuckle Tea

The other day, while strolling through our neighborhood, I came across large swath of greenery dotted with delightful flowers:



My first instinct was to snap pics. Which I did. A lot. Then I turned to new app I've been playing with: Plant Snap. Plant Snap identifies plants from a picture you take within the app.

The concept reminds me of this classic commic:

To my shock, the app works surprisingly well. I tried it on a number of known plants and it properly identified most of them.  On unknown plants, it has served to give me a solid starting point. The app isn't magic and I wouldn't trust it blindly. But as a tool for the amateur botanist, especially one who doesn't want to nag others to learn about plants, the app appears to be a winner.

At $9.99/year, Plant Snap isn't cheap. But it has one notable feature which may justify the price: if automatic plant ID fails you can supposedly get a knowledgeable human to step in and help.

Back in front of the pretty flowers I snapped a pic with PlantSnap:

I was looking at Japenese Honeysuckle. Follow up research showed this to be true. I'm still amazed that the app guessed this from a single picture of a leaf.

Reading up on honeysuckle, it's a wonder  I didn't instantly recognize the plant myself. It's one of those classic plants that was brought to the US with such promise, and then proceeded to wreak utter havoc. While the plant is medicinally promising, pretty and easy to grow, it's fast growth and habit of strangling other plants means it's often considered an invasive weed.

Honeysuckle smells delightful, and a quick Google search revealed that you can make tea from the flowers. Which naturally I had to do. The stand of honeysuckle I found was on public land and plentiful, so I had no qualms about picking a few flowers to give this tea a try.

Preparation was simple: I put the flowers in a small glass jar, added boiling water and waited. I ran the drink through a strainer and found myself with a warm cup of yellow-green tinted liquid. I took a sip: it tasted flowery, woody and leafy. It wasn't love a first sip, but it wasn't bad either. On a backpacking trip, where everything tastes better, I bet having some fresh honeysuckle tea would be a nice treat.



I've always tried to treat my local environment as a sort of treasure hunting grounds: the gems are there, you just need to look past the see-it-everyday blindess that sets in. Plant Snap is the perfect tool for embracing this philosophy. Every random green thing in your neighborhood is now a click away from revealing its identity. So get snapping.

Friday, June 05, 2020

A Most Unusual Bedtime Playlist

Picking music for your baby's bedtime is easy. Fire up Soma FM's Groove Salad and you've got the perfect environment to send your little one off to dream land.

Except our little guy didn't get the memo. When he gets overtired and both desperately needs and fights sleep the ambient beats over at Groove Salad do nothing. Classical Music, Chill and Post Rock all come up equally short.

What he needs, and it's beyond counterinutive, is to rock out to some bang'n tunes. I wouldn't believe it except I witness this on a daily basis. To watch our little one drift off to sleep while Dwight Yokum or Bishop Briggs loudly jams away is to witness a mystery and miracle.

So here it is, the world's most unusual bedtime music list. Kids are weird.

Sweet Dreams!

Monday, June 01, 2020

From Ferocious Meat-Eater to Handsome Acrobat

Meet Ed the Eyed Elater. Ed is a handsome beetle with a passion for acrobatics (he can flip himself off his back, hurling himself four times his body length). But don't let Ed's good looks and clever party trick fool you. In larvae form, he's a beast:

Found under logs and other dark, damp places, the Alaus oculatus larva looks like a stocky, yellowish-brown, segmented worm. It has a flat, dark brown rectangular head that ends in 2 powerful jaws. The jaws, which resemble small crab legs, are used to disable and dismember prey. An individual is about 2 inches long. It looks rather dangerous at the posterior end, too. The 10th segment has 2 anal hooks, 10-12 spines, and setae (hairs) in front of the anus.

Fortunately, Ed puts his powerful jaws, anal hooks and spines to good use eating garden pests. So if you see Ed, a Thank You For Your Service is in order.

Mostly I'm glad Ed stopped by my porch so I could meet a new critter all without leaving the comfort of our quarantine.