Showing posts with label scheme. Show all posts
Showing posts with label scheme. Show all posts

Wednesday, July 17, 2019

Don't Fear the x, and other lessons from Shamir Secret Sharing

Consider these facts about this 3rd degree polynomial:

f(x) = 1822 + 3x + 19x2 + 2x3

Fact 1. This is considered a 3rd degree polynomial because the largest exponent value is 3. While looking at this equation gives me a burst of High School Math PTSD, it need not. Upon further reflection, it's just a bit of terse code. I could program the above as:

(define f (lambda (x)
            (+ 1822 
               (* 3 x)
               (* 19 (expt x 2))
               (* 2  (expt x 3)))))

Because all polynomials have the same shape, it's easy to make a function that generates polynomials:

(define (make-poly coeffs)
  (lambda (x)
    (let loop ((coeffs coeffs)
               (i 0)
               (sum 0))
      (cond ((null? coeffs) sum)
            (else
             (loop (cdr coeffs)
                   (+ i 1)
                   (+ sum (* (car coeffs) (^ x i)))))))))

With this 'make' function, I can replace the above code with the following:

(define f (make-poly '(1822 3 19 2)))

I can call f with as many values of x as I wish:

> (for-each (lambda (x) (show (cons x (f x)))) (range 0 10))
((0 . 1822))
((1 . 1846))
((2 . 1920))
((3 . 2056))
((4 . 2266))
((5 . 2562))
((6 . 2956))
((7 . 3460))
((8 . 4086))
((9 . 4846))
((10 . 5752))

Fact 2. Evaluating a polynomial at x = 0 always returns the value of the first constant term. We see this above, as 0 maps to 1822. This will continue to hold true no matter how ugly and complex the polynomial is.

Fact 3. Using Lagrange Interpolating polynomials we can use degree + 1 values of a polynomial to construct a function which will return values equivalent to the original polynomial. That is, if we feed any four values above into make-lagr-poly we end up with a new function fg which computes the same values as f.

(define fg (make-lagr-poly '((3 . 2056)
                            (6 . 2956)
                            (7 . 3460)
                            (10 . 5752))))

> (for-each (lambda (x) (show (cons x (fg x)))) (range 0 10))
((0 . 1822))
((1 . 1846))
((2 . 1920))
((3 . 2056))
((4 . 2266))
((5 . 2562))
((6 . 2956))
((7 . 3460))
((8 . 4086))
((9 . 4846))
((10 . 5752))

The how and why of Lagrange polynomials will have to wait for another blog post. But for now, trust me that this bit of math-magic works.

Cryptographer Adi Shamir combined the above facts to create something quite useful: a way to securely share a secret among a group.

Consider this problem:

You need to distribute a vault code to bank executives, however you'd like to avoid a number of pitfalls. For one, you want to make sure that a single lost or stolen code doesn't allow an attacker into the vault. You'd also like to insure that at least 4 executives need to agree on the requirement of opening the vault before it can be accessed. This reduces the chances that a small number of executives will coordinate to steal from the vault.

Shamir used the above math to create Shamir Secret Sharing. Here's how it works. First, note your secret, which in this case is the vault code. We'll use 12345 as our code. Then decide on your threshold, that is the number of executives that are required before the secret can be revealed. We'll use 4 as suggested above. Finally, decide on how many shares you want to give out. Let's assume there are 10 executives, so we'll generate 10 shares.

Step one is to form a polynomial of threshold - 1 degree, where the first term is the secret:

;; values 4, 19 and 103 are random numbers, and should be
;; generated in a secure and truly random way.
(define secret (make-poly '(12345 4 19 103))) 

Step two is to generate the 10 shares, one for each executive:

> (for-each (lambda (x) (show (cons x (secret x)))) (range 1 10))
((1 . 12471))
((2 . 13253))
((3 . 15309))
((4 . 19257))
((5 . 25715))
((6 . 35301))
((7 . 48633))
((8 . 66329))
((9 . 89007))
((10 . 117285))

Note how we start x at 1 and not 0. Calling this function with 0 reveals our secret.

We give each pair of numbers to an executive and instruct them to keep the pair private. If an attacker obtains less than 4 shares, then the secret remains safe and no clues about the it are revealed.

When the vault needs to be opened, 4 executives pool their shares to calculate a function equivalent to the original polynomial:

(define soln (make-lagr-poly '((1 . 12471)
                               (5 .  25715)
                               (9 . 89007)
                               (10 . 117285))))

Finally, to derive the secret we pass 0 to the solution function:

> (soln 0)
12345

While the above implementation captures the essence of Shamir Secret Sharing, it's missing an important step to make it truly secure. I may cover that in a future post.

I had two takeaways from my experience implementing Shamir Secret Sharing. First, it's fascinating to see how mathematical properties can be combined to solve everyday problems

And Second, I found the math behind this scheme to be initially quite overwhelming. I struggled to both understand the mathematical notation, as well as how to convert it to code. And then it hit me: I've got a programming language that let's me directly implement polynomial and Lagrange interpolations functions; I don't need to think of them as purely symbolic expressions like so many of the examples on the web do. Putting this in terms of code and not math made all the difference for me.

What a joy it was to untangle what initially appeared to be so complex.

Find my implementation of Shamir Secret Sharing here. Find the original paper by Adi Shamir here. Enjoy!

Friday, April 27, 2018

Elise Four - Strong, Human Powered, Encryption

If you want to understand a subject, teach it to someone. If you want to truly and completely understand a subject, teach it to a computer. That is, program it.

This idea of coding to reach understanding is a lesson I've learned countless times. Most recently, I experienced this adage while tackling the Elise Four exercise from Programming Praxis. The task was to implement the EliseFour (aka, LC4) Encryption Algorithm. This approach to encryption is especially interesting because of its competing goals. First, it attempts to provide for strong encryption. Secondly, it's designed to be run offline by human power alone. (Not unlike this solution)

See the contradiction there? All the tools one thinks of when considering strong encryption, such as terrifically hard calculations and massively large numbers, are off the table. LC4 therefor has to choose clever conventions over raw processing power. The author solves this challenge by constructing a grid of Scrabble-like tiles and re-arranging them using a simple set of rules. While I'm interesting in implementing an offline version of the algorithm, for now, I've coded the algorithm in Scheme.

Here's the algorithm at work:

;; Example 1
Key:  xv7ydq #opaj_ 39rzut 8b45wc sgehmi knf26l 
Nonce:  solwbf 
Text:  im_about_to_put_the_hammer_down 
Encrypted:  i2zqpilr2yqgptltrzx2_9fzlmbo3y8_9pyssx8nf2 
Decrypted:  im_about_to_put_the_hammer_down#rubberduck 

;; Example 2
Key:  xv7ydq #opaj_ 39rzut 8b45wc sgehmi knf26l 
Nonce:  argvpx 
Text:  hurrrraaaaaaaaaaaaaaaaaaaaaay 
Encrypted:  3bcxnt57hus6accn97v4iie__hjbml8wr 
Decrypted:  hurrrraaaaaaaaaaaaaaaaaaaaaay#ben 

The source code for my solution can be found in the usual spot.

Example 1 comes directly from the paper describing the algorithm and was my lifeline for implementing and verifying the algorithm. You can see that to communicate with the LC4 algorithm, both users need to know the key value ahead of time. The key will always be an arrangement of the 36 different characters in the alphabet.

Encryption and decryption also depend on a nonce value, which is used to add additional noise into the system. The nonce value is to be shared with the recipient, and no harm comes from it being exposed to an attacker.

Example 2 emphasizes that even text with a telling shape comes out as random gibberish when encrypted; a sign that his is indeed a secure algorithm.

LC4 is an elegant solution to an unusual encryption challenge. On one level, I knew this by browsing the source paper on the topic. But now that I've struggled through an implementation, I know this on a completely different level. And when I say struggle, I mean it. Encryption algorithms are tricky to implement because the goal is to take text and turn it into garbage, but very specific garbage. So when I had variable c where y belonged, or neglected to reset r and y the algorithm still produced jumbled text, but it was the wrong jumbled text.

At least four times I was absolutely sure I had coded the encryption and decryption algorithm exactly as described, yet my code was failing. I did what any smart programmer does in these situations: I stepped away and re-attacked the problem after some rest. With some time away from the code, the bugs were obvious and easy to address.

When I finished coding the algorithm I had far more than just working code. I had an intimate appreciation for key generation, nonce usage, dictionary selection, authentication operation and a half dozen other subtle details that work together to make LC4 function. Oh, and the dopamine rush from seeing my text get encrypted and then decrypted was nice too.

Tuesday, April 24, 2018

Lightweight, R7RS Compliant and in palm of my hand - Compiling chibi-scheme on Android

I'm eyeing this Programming Praxis exercise as one to tackle next but there's a catch before I can start programming. TinyScheme, my current Scheme Interpreter of choice, doesn't have support for generating random numbers, and the exercise requires this.

I could hack something together using TinyScheme (and probably the Programming Praxis Standard Prelude). Or, I could do as John Cowen suggested, and try out a different implementation. He suggested Chibi-Scheme. Chibi Scheme is lightweight, but unlike TinyScheme, it's also quite modern. This means, among other things, it should give me easy access to the SRFI-27: Sources of Random Bits library. Longer term, it would be ideal to have access to R7RS features and other modern niceties.

I busted out my cell phone and hardware keyboard. I opened up Termux and went to work. Here's my attempt at getting chibi-scheme running on Android:

Let's be clever and build this on the SD card.

[localhost]{~}(1)$ cd storage/external-1/src

[localhost]{src}(1)$ git clone https://github.com/ashinn/chibi-scheme.git
Cloning into 'chibi-scheme'...
remote: Counting objects: 18101, done.
remote: Compressing objects:  ....
Receiving objects: 100% 
Resolving deltas:   0% 

[localhost]{src}(1)$ cd chibi-scheme

[localhost]{chibi-scheme}(1)$ ./configure
bash: ./configure: /bin/sh: bad interpreter: Permission denied

D'oh. OK, don't panic.

[localhost]{chibi-scheme}(1)$ sh configure
Autoconf is an evil piece bloatware encouraging cargo-cult programming.
Make, on the other hand, is a beautiful little prolog for the filesystem.
Just run 'make'.

Oooh, sassy software. I like it.

[localhost]{chibi-scheme}(1)$ make
The program 'make' is not installed. Install it by executing:
 pkg install make

No problem, I'll just do what Termux tells me to do...

[localhost]{chibi-scheme}(1)$ pkg install make
Building dependency tree...
Reading state information...
All packages are up to date.
The following packages were automatically installed and are no longer required:
  binutils libandroid-support-dev libffi libllvm ndk-stl ndk-sysroot
Use 'apt autoremove' to remove them.
The following NEW packages will be installed:
  make
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 76.5 kB of archives.
After this operation, 246 kB of additional disk space will be used.
Get:1 https://termux.net stable/main aarch64 make aarch64 4.2.1 [76.5 kB]
Preparing to unpack .../make_4.2.1_aarch64.deb ...



[localhost]{chibi-scheme}(1)$ make
echo '#define sexp_so_extension "'.so'"' > include/chibi/install.h
echo '#define sexp_default_module_path "'/data/data/com.termux/files/usr/share/chibi:/data/data/com.termux/files/usr/lib/chibi'"' >> include/chibi/install.h
echo '#define sexp_platform "'android'"' >> include/chibi/install.h
echo '#define sexp_version "'0.8.0'"' >> include/chibi/install.h
echo '#define sexp_release_name "'`cat RELEASE`'"' >> include/chibi/install.h
cc -c  -Iinclude  -Wall -g -g3 -O3  -o main.o main.c
make: cc: Command not found
make: *** [Makefile:149: main.o] Error 127

Whoops. No compiler. After attempts to install 'gcc' and search for a compiler
package in general I finally tripped over 'clang.' That will apparently install
a C compiler.

[localhost]{chibi-scheme}(1)$ pkg install clang
The following NEW packages will be installed:
  clang
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 12.9 MB of archives.
After this operation, 61.2 MB of additional disk space will be used.
Get:1 https://termux.net stable/main aarch64 clang aarch64 6.0.0-1 [12.9 MB]
Preparing to unpack .../clang_6.0.0-1_aarch64.deb ...

[localhost]{chibi-scheme}(1)$ make
cc -c -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3  -o main.o main.c
cc -c -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3  -fPIC -o gc.o gc.c


It's working! It's really working!

cc -c -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3  -fPIC -o sexp.o sexp.c
cc -c -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3  -fPIC -o bignum.o bignum.c
cc -c -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3  -fPIC -o gc_heap.o gc_heap.c
cc -c -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3  -fPIC -o opcodes.o opcodes.c
cc -c -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3  -fPIC -o vm.o vm.c
cc -c -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3  -fPIC -o eval.o eval.c
cc -c -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3  -fPIC -o simplify.o simplify.c
cc -fPIC -shared -Wl,-soname,libchibi-scheme.so.0 -o libchibi-scheme.so.0.8.0 gc.o sexp.o bignum.o gc_heap.o opcodes.o vm.o eval.o simplify.o    -ldl -lm
ln -sf libchibi-scheme.so.0.8.0 libchibi-scheme.so.0
ln: libchibi-scheme.so.0: Operation not permitted
make: *** [Makefile:162: libchibi-scheme.so.0] Error 1

And it broke. Ugh.  After a much debugging I realized that my issue
was permission based. Apparently Android doesn't allow you to perform
certain operations on your SD card. I moved the source tree to internal 
storage and tried again.

[localhost]{chibi-scheme}(1)$ cd ..
[localhost]{src}(1)$ mv chibi-scheme ~/

[localhost]{src}(1)$ cd

[localhost]{~}(1)$ cd chibi-scheme
[localhost]{chibi-scheme}(1)$ make
ln -sf libchibi-scheme.so.0.8.0 libchibi-scheme.so.0
ln -sf libchibi-scheme.so.0 libchibi-scheme.so

It's working again. Whooo!

cc -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o chibi-scheme main.o -L. -lchibi-scheme
LD_LIBRARY_PATH=".:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH=".:" CHIBI_MODULE_PATH=lib ./chibi-scheme -q tools/chibi-ffi lib/chibi/filesystem.stub
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/filesystem.so lib/chibi/filesystem.c -L.  -lchibi-scheme
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/weak.so lib/chibi/weak.c -L.  -lchibi-scheme
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/heap-stats.so lib/chibi/heap-stats.c -L.  -lchibi-scheme
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/disasm.so lib/chibi/disasm.c -L.  -lchibi-scheme
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/ast.so lib/chibi/ast.c  -L. -lchibi-scheme
LD_LIBRARY_PATH=".:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH=".:" CHIBI_MODULE_PATH=lib ./chibi-scheme -q tools/chibi-ffi lib/chibi/emscripten.stub
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/emscripten.so lib/chibi/emscripten.c -L.  -lchibi-scheme
LD_LIBRARY_PATH=".:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH=".:" CHIBI_MODULE_PATH=lib ./chibi-scheme -q tools/chibi-ffi lib/chibi/process.stub
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/process.so lib/chibi/process.c -L.  -lchibi-scheme
LD_LIBRARY_PATH=".:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH=".:" CHIBI_MODULE_PATH=lib ./chibi-scheme -q tools/chibi-ffi lib/chibi/time.stub
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/time.so lib/chibi/time.c -L.  -lchibi-scheme
LD_LIBRARY_PATH=".:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH=".:" CHIBI_MODULE_PATH=lib ./chibi-scheme -q tools/chibi-ffi lib/chibi/system.stub
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/system.so lib/chibi/system.c -L.  -lchibi-scheme
LD_LIBRARY_PATH=".:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH=".:" CHIBI_MODULE_PATH=lib ./chibi-scheme -q tools/chibi-ffi lib/chibi/stty.stub
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/stty.so lib/chibi/stty.c -L.  -lchibi-scheme
LD_LIBRARY_PATH=".:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH=".:" CHIBI_MODULE_PATH=lib ./chibi-scheme -q tools/chibi-ffi lib/chibi/net.stub
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/net.so lib/chibi/net.c -L.  -lchibi-scheme
LD_LIBRARY_PATH=".:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH=".:" CHIBI_MODULE_PATH=lib ./chibi-scheme -q tools/chibi-ffi lib/chibi/io/io.stub
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/io/io.so lib/chibi/io/io.c -L.  -lchibi-scheme
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/optimize/rest.so lib/chibi/optimize/rest.c -L.  -lchibi-scheme
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/optimize/profile.so lib/chibi/optimize/profile.c -L.  -lchibi-scheme
LD_LIBRARY_PATH=".:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH=".:" CHIBI_MODULE_PATH=lib ./chibi-scheme -q tools/chibi-ffi lib/chibi/crypto/crypto.stub
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/chibi/crypto/crypto.so lib/chibi/crypto/crypto.c -L.  -lchibi-scheme
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/srfi/27/rand.so lib/srfi/27/rand.c -L.  -lchibi-scheme
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/srfi/151/bit.so lib/srfi/151/bit.c -L.  -lchibi-scheme
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/srfi/39/param.so lib/srfi/39/param.c -L.  -lchibi-scheme
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/srfi/69/hash.so lib/srfi/69/hash.c -L.  -lchibi-scheme
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/srfi/95/qsort.so lib/srfi/95/qsort.c -L.  -lchibi-scheme
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/srfi/98/env.so lib/srfi/98/env.c -L.  -lchibi-scheme
LD_LIBRARY_PATH=".:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH=".:" CHIBI_MODULE_PATH=lib ./chibi-scheme -q tools/chibi-ffi lib/srfi/144/math.stub
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/srfi/144/math.so lib/srfi/144/math.c -L.  -lchibi-scheme
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/scheme/time.so lib/scheme/time.c -L.  -lchibi-scheme
cc -fPIC -shared -DSEXP_USE_INTTYPES -Iinclude  -Wall -g -g3 -O3   -o lib/srfi/18/threads.so lib/srfi/18/threads.c -L.  -lchibi-scheme
echo "# pkg-config" > chibi-scheme.pc
echo "prefix=/data/data/com.termux/files/usr" >> chibi-scheme.pc
echo "exec_prefix=\${prefix}" >> chibi-scheme.pc
echo "libdir=/data/data/com.termux/files/usr/lib" >> chibi-scheme.pc
echo "includedir=\${prefix}/include" >> chibi-scheme.pc
echo "version=0.8.0" >> chibi-scheme.pc
echo "" >> chibi-scheme.pc
cat chibi-scheme.pc.in >> chibi-scheme.pc
find lib/chibi/ -name \*.sld | \
 LD_LIBRARY_PATH=".:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH=".:" CHIBI_MODULE_PATH=lib ./chibi-scheme tools/generate-install-meta.scm 0.8.0 > lib/.chibi.meta
find lib/srfi/ -name \*.sld | \
 LD_LIBRARY_PATH=".:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH=".:" CHIBI_MODULE_PATH=lib ./chibi-scheme tools/generate-install-meta.scm 0.8.0 > lib/.srfi.meta
find lib/scheme/ -name \*.sld | \
 LD_LIBRARY_PATH=".:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH=".:" CHIBI_MODULE_PATH=lib ./chibi-scheme tools/generate-install-meta.scm 0.8.0 > lib/.scheme.meta

Amazing - it built without issue. How about installing, will that work?

[localhost]{chibi-scheme}(1)$ make install
install -d /data/data/com.termux/files/usr/bin
install -m0755 chibi-scheme /data/data/com.termux/files/usr/bin/
install -m0755 tools/chibi-ffi /data/data/com.termux/files/usr/bin/
install -m0755 tools/chibi-doc /data/data/com.termux/files/usr/bin/
install -m0755 tools/snow-chibi /data/data/com.termux/files/usr/bin/
install -m0755 tools/snow-chibi.scm /data/data/com.termux/files/usr/bin/

many boring 'install' lines removed. You're welcome.

if type ldconfig >/dev/null 2>/dev/null; then ldconfig; fi
echo "Generating images"
Generating images
cd / && LD_LIBRARY_PATH="/data/data/com.termux/files/usr/lib:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH="/data/data/com.termux/files/usr/lib:" /data/data/com.termux/files/usr/bin/chibi-scheme -d /data/data/com.termux/files/usr/share/chibi/chibi.img
cd / && LD_LIBRARY_PATH="/data/data/com.termux/files/usr/lib:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH="/data/data/com.termux/files/usr/lib:" /data/data/com.termux/files/usr/bin/chibi-scheme -xscheme.red -d /data/data/com.termux/files/usr/share/chibi/red.img
cd / && LD_LIBRARY_PATH="/data/data/com.termux/files/usr/lib:/data/data/com.termux/files/usr/lib" DYLD_LIBRARY_PATH="/data/data/com.termux/files/usr/lib:" /data/data/com.termux/files/usr/bin/chibi-scheme -mchibi.snow.commands -mchibi.snow.interface -mchibi.snow.package -mchibi.snow.utils -d /data/data/com.termux/files/usr/share/chibi/snow.img

Holy smokes, it's done. Let's see if it really worked.

[localhost]{chibi-scheme}(1)$ cd

[localhost]{~}(1)$ chibi-scheme
> (+ 1 2 3)
6

We have scheme! Let's try something a bit more advanced

> (import (srfi 27) (srfi 1))
> (random-integer 100)
56
> 

The R7RS library functionality is working and 
srfi's are installed by default. We have success!

And here are some screenshots to show this really was all executed on my Galaxy S9 Plus:

Monday, April 23, 2018

A Little Scheme Setup and Development on the Galaxy S9 Plus

Today's goal was to setup a Scheme programming environment on my new Galaxy S9 Plus phone. I planned to use the same tools I'd used on my LG G6:

  • Termux - provides a Unixy environment, doesn't require root and is just plain awesome
  • tinyscheme - provides a lightweight, yet powerful, scheme environment. For bonus points, it's trivial to install within Termux. Just type: pkg install tinyscheme
  • emacs, vim, git - these tools run well under Termux and are installed using the pkg install command. I use the same emacs config on my phone as I do my desktop.
  • External Keyboard Helper - remaps keys on my hardware keyboard. Specifically, I use it to map Caps Lock to Escape. I've found that this keybinding is essential to making bash, emacs and vim work in a sane way.

With the tools installed, I was able to open up a new .scm file in emacs, type C-u M-x run-scheme tinyscheme and I was off and running with an interactive Scheme development environment.

While the tools installed without issue, I couldn't consider the environment setup until I wrote some code. So off I went to ProgrammingPraxis.com to find an interesting exercise to tackle.

Sticking with the theme of rendering text in unique ways, I decided to complete the Seven-Segment Devices exercise. A Seven-Segment Device is a primitive display that renders numbers and text using seven different lit up bars. Think old school calculators. Here are the seven different segments you have to work with:

 --2--
|     |
3     4
|     |
 --1--
|     |
5     6
|     |
 --0--

Below you'll find my solution that turns any integer into text that's rendered using this seven-segment style. The exercise called for storing the data as a bit-array. I took an alternative approach, and mapped every digit in the number to a list of enabled segments. For example, 0 maps to (2 4 6 0 5 3). This is more verbose than packing the data into bits, but I was more interested in rendering the text than storing a compact representation of it.

Here are some screenshots of the code running, and the code itself is below. It felt good to be coding on my new phone, and it sure is remarkable how well all these tools Just Work.

;; https://programmingpraxis.com/2018/02/27/seven-segment-devices/


(define (show . args)
  (display args)
  (newline))

(define (range lower upper)
  (if (< lower upper)
    (cons lower (range (+ 1 lower) upper))
    '()))

(define *digit-width* 10)
(define *digit-height* 11)
(define *digit-sep* 4)

(define *digit-map*
  '((0 . (2 4 6 0 5 3))
    (1 . (4 6))
    (2 . (2 4 1 5 0))
    (3 . (2 4 1 6 0))
    (4 . (3 1 4 6))
    (5 . (2 3 1 6 0))
    (6 . (3 5 0 6 1))
    (7 . (2 4 6))
    (8 . (0 1 2 3 4 5 6))
    (9 . (2 4 1 3 6))))

(define (integer->bars val)
  (if (= val 0)
    (cdr (assoc 0 *digit-map*))
    (let loop ((val val) (bars '()))
      (cond ((<= val 0) 
             bars)
            (else
             (let ((d (modulo val 10)))
               (loop
                (inexact->exact (floor (/ val 10)))
                (cons (cdr (assoc d *digit-map*)) bars))))))))
  

(define (draw-sep)
  (for-each (lambda (i)
              (display " "))
            (range 0 *digit-sep*)))

(define (draw first mid last)
  (display first)
  (for-each (lambda (i)
              (display mid))
            (range 0 (- *digit-width* 2)))
  (display last))

(define (draw-row digits row)
  (define mid (/ (- *digit-height* 1) 2))
  (define top? (= row 0))
  (define bottom? (= row (- *digit-height* 1)))
  (define mid? (= row mid))
  (define upper? (and (not top?) (< row mid)))
  (define lower? (and (not bottom?) (> row mid)))
  (for-each (lambda (bars)
              (define (has? x) (member x bars))
              (cond ((and top? (has? 2))
                     (draw "+" "-" "+"))
                    ((and bottom? (has? 0))
                     (draw "+" "-" "+"))
                    ((and mid? (has? 1))
                     (draw "+" "-" "+"))
                    ((and upper? (has? 3) (has? 4))
                     (draw "|" " " "|"))
                    ((and upper? (has? 3))
                     (draw "|" " " " "))
                    ((and upper? (has? 4))
                     (draw " " " " "|"))
                    ((and lower? (has? 5) (has? 6))
                     (draw "|"   " "   "|"))
                    ((and lower? (has? 5))
                     (draw "|" " " " "))
                    ((and lower? (has? 6))
                     (draw " " " " "|"))
                    (else
                     (draw " " " " " ")))
              (draw-sep))
            digits)
  (newline))


(define (draw-integer x)
  (define s (integer->bars x))
  (newline)
  (for-each (lambda (row)
              (draw-row s row))
            (range 0 *digit-height*))
  x)

Wednesday, December 20, 2017

Helping bash and tinyscheme get along | Friendlier command line execution of scheme code

I was looking for a cleaner way to manage SDR Touch radio frequencies and I thought I'd give Scheme S-expressions a go.

I quickly arrived at this 'format':


(("Arl,VA"
  (166.9 "DC Police?")
  (166.7 "Dc Police?"))

 ("OnABoat"
  (462 "FRS Low")
  (467 "FRS High")
  (457.5 "Marine1")
  (467.5 "Marine2")
  (467.7 "Marine3"))

 ("NCL"
  (457.5250 "Bridge Ops")
  (457.5500 "Engineering")
  (457.5750 "???"))

Because I programmed the solution in Scheme, turning this basic set of S-expressions into the required mess of XML was straightforward. You can find the solution here. I have to say, it was a real delight using S-Expressions to solve this one. XML, JSON and raw text all jumped out at me as options, but in this case, S-Expressions were the clear way to go.

I coded this solution on my Android device using Termux, emacs and Tinyscheme.

While the solution worked great under emacs' interactive scheme mode, I decided I wanted the ability to run the conversion command from a bash prompt. While tinyscheme has some basic command line handling, it was lacking a few details, such as the ability to get the script's working directory. This may seem like an obscure requirement, but with this value I'm able to reliably (load ...) files without worrying about running the script from a specific directory.

To make tinyscheme more command line friendly, I created tsexec, a small script below that kicks off tinyscheme with a number of helpful functions predefined. These include:

(cmd-arg0) - returns the name of scheme script that is currently executing.

(cmd-args) - returns the list of command line arguments passed to the currently executing script.

(cmd-dir . path) - when invoked with no arguments, returns the directory the currently executing script lives in. When an argument is provided, the value is prefixed with the source script's directory.

Using these functions I'm was able to write a command line friendly version of my scheme code to generate XML.

(load (cmd-dir "utils.scm"))  ;; utils.scm is now imported in a location independent way

(define (make-presets frequency-file xml-file)
  (with-output-to-file xml-file
    (lambda ()
      (with-data-from-file frequency-file
       (lambda (data)
  (show "<?xml version='1.0' encoding='UTF-8'?>")
  (show "<sdr_presets version='1'>")
  (for-each (lambda (category)
       (show "<category id='" (gen-id) "' name='" (car category) "'>")
       (for-each (lambda (preset)
     (show "<preset id='" (gen-id) "' ")
     (show "        name='" (cadr preset) "' ")
     (show "        freq='" (freqval (car preset)) "' ")
     (show "        centfreq='" (freqval (car preset)) "' ")
     (show "        offset='0'")
     (show "        order='" (gen-id) "' ")
     (show "        filter='10508' ")
     (show "        dem='1' ")
     (show "/>"))
          (cdr category))
       (show "</category>"))
     data)
  (show "</sdr_presets>"))))))
  

;; Simple argument handling below
(cond
 ((= 1 (length (cmd-args)))
  (make-presets (car (cmd-args)) "SDRTouchPresets.xml"))
 ((= 2 (length (cmd-args)))
  (make-presets (car (cmd-args)) (cadr (cmd-args))))
 (else
  (show "Need to provide frequency and output file to continue.")))

Here's a few screenshots of me running this code:

Termux has a handy add-on that allows you to invoke shell scripts from your phone's home screen. I'm thinking tsexec may be an ideal way to connect up this short-cut ability to tinyscheme.

Finally, here's the source code for tsexec. Enjoy!

#!/data/data/com.termux/files/usr/bin/bash

##
## Execute a tinyscheme script. Setup various functions that can
## be used to access command line args and such
##

if [ ! -f "$1" ] ; then
    echo "Refusing to run non-existant script [$1]"
    exit 1
fi

CMD_DIR=$(pwd)/$(dirname $1)
CMD_ARG0=$(basename $1)
CMD_MAIN="$CMD_DIR/$CMD_ARG0"
shift

CMD_ARGS=""
for arg in "$@" ; do
  CMD_ARGS="$CMD_ARGS \"$arg\""
done


if [ ! -f "$CMD_MAIN" ] ; then
    echo "Failed to derive CMD_DIR, guessed: $CMD_DIR"
    exit 2
fi

wrapper() {
  cat <<EOF
;; Autogenerated scheme wrapper script
(define (cmd-arg0) 
  "$CMD_ARG0")

(define (cmd-dir . path)
  (if (null? path)
      "$CMD_DIR"
      (string-append "$CMD_DIR/" (car path))))

(define (cmd-args)
  (list $CMD_ARGS))

(load "$CMD_MAIN")
EOF
}

wrapper > $HOME/.ts.$$
tinyscheme $HOME/.ts.$$
rm $HOME/.ts.$$

Monday, November 13, 2017

Hop to it - Counting Jumps using a cell phone accelerometer

Last week I was harshly schooled in the difference between theoretical physics and the reality of cell phone hardware. While it may be true that you can calculate distance from acceleration, the hardware on the LG G6 isn't precise enough to do so. But I remain undeterred!

I return instead, to my original and simpler problem: can I use my phone's sensors to determine many jumps have I done in a jump rope session? The accelerometer is clearly one way to approximate this. Just check out the graph below. It shows three bursts of me jumping, with each set of jumps followed by a pause:

The simplest strategy here is to count the peaks of the graph. Swapping out the code to calculate distance with code to do this counting was simple enough. All the code can be found here, but the relevant lines as follows:

;; Counting the peaks in a given accelerometer stream.  This should
;; give us a rough approximation of jumps
(define (count-peaks accum data)
  (let ((lower 0)
 (upper 10)
 (verbose? #t)
 (a-now (+ (data 1) (data 2) (data 3)))
 (rising? (accum 0))
 (num-peaks (accum 1)))
    (cond ((and rising? (>= a-now upper))
    (if verbose?
        (show "peak discovered: " a-now))
    (list #f (+ 1 num-peaks)))
   ((and (not rising?) (<= a-now lower))
    (list #t num-peaks))
   (else
    (list rising? num-peaks)))))

For now, a peak is defined as having total acceleration over 10m/s2. I arrived at this number the way all important mathematical constants are: I stood in my living room and jumped up and down. Set the peak height too low, and simple movements are marked as jump. Set it too high, and not all the jumps are counted.

A smarter approach would be to make this number adaptive, making it somehow dependent on the data set itself. But for now, a constant of 10 seems to work OK.

With this code change, I was able to record a 10 jump data file and run it through tinyscheme:

While this was functional, I was curious if I could streamline running this script. Switching to Termux, then entering the right filename in emacs, and then evaluating the code in the *scheme* buffer was all a bit much.

Fortunately, Termux has the answer: Termux:Task. This is an add on for Tasker that allows you to kick off shell scripts under Termux. The first order of business was to wrap up the scheme code as a shell script. That wasn't hard:

#!/data/data/com.termux/files/usr/bin/sh

SRC_DIR=$HOME/dt/i2x/code/android-accelerometer-playtime/
MAIN_SCM=$SRC_DIR/main.scm

usage () {
    echo "Usage: `basename $0` /path/to/data.csv"
    exit
}

if [ -f "$1" ] ; then
    exec tinyscheme -c "(define *src-dir* \"$SRC_DIR\") (load \"$MAIN_SCM\") (show (count-jumps \"$1\"))"
else
    usage
fi

tinyscheme can be invoked with -c which allows for running arbitrary scheme expressions. I'm using this to initialize and run parameterized code.

From there, I soft linked this script under $HOME/.termux/tasker. This allowed the script to be visible inside of Tasker:

Note the use of %asfile1. This variable is set by AutoShare, a Tasker plugin that lets you kick off actions based on the system sharing menu.

With the Tasker code in place, I can record data in the Physics Toolbox Sensor Suite app, then share that data with the Count Jumps AutoShare command. This kicks off the count_jumps shell script, which invokes tinyscheme and runs my code above. Simple, right? It may be a virtual Rube Goldberg, but it does work, and successfully demonstrates how you can get from an Android App to scheme code with very little effort.

As a final test, here's me knocking out 30 test jumps:

And there you have it, 31 jumps. As approximations go, I'll take it.

Thursday, June 16, 2016

Got Data? Adventures in Virtual Crystal Ball Creation

There's two ways to look at this recent Programming Praxis exercise: implementing a beginner level statistics function or creating a magical crystal ball that can predict past, present and future! I chose to approach this problem with the mindset of the latter. Let's make some magic!

The algorithm we're tackling is linear regression. I managed to skip statistics in college (what a shame!), so I don't recall ever being formally taught this technique. Very roughly, if you have the right set of data, you can plot a line through it. You can then use this line to predict values not in the data set.

The exercise gave us this tiny, manufactured, data set:

x    y
60   3.1
61   3.6
62   3.8
63   4.0
65   4.1

With linear regression you can answer questions like: what will be the associated value for say, 64, 66 or 1024 be? Here's my implementation in action:

A few words about the screenshot above. You'll notice that I'm converting my data from a simple list to a generator. A generator in this case is a function that will return a single element in the data set, and returns '() when all the data has been exhausted. I chose to use a generator over a simple list because I wanted to allow this solution to scale to large data sets.

Below you'll see a data set that's stored in a file and leverages a file based generator to access its contents. So far, I haven't throw a large data set at this program, but I believe it should scale without issue.

The call to make-crystal-ball performs the linear-regression and returns back a function that when provided x returns a guess prediction for y. What? I'm trying to have a bit of fun here.

Looking around on web I found this example that uses a small, but real data set. In this case, it compares High School and College GPA. Using linear-regression we're able to predict how a High School student with a 2.7, 3.0 or 3.5 GPA is going to do in college. Here's the code:

;;  http://onlinestatbook.com/2/regression/intro.html
;;  http://onlinestatbook.com/2/case_studies/sat.html
(define (gpa-sat-test)
  (define data (file->generator "gpa-sat-data.scm"
                                (lambda (high-gpa math-sat verb-sat comp-gpa univ-gpa)
                                  (list high-gpa univ-gpa))))
  (let ((ball (make-crystal-ball data)))
    (show "2.7 => " (ball 2.7))
    (show "3.0 => " (ball 3))
    (show "3.5 => " (ball 3.5))))

And the answer is: 2.91, 3.12 and 3.45 respectively. So yeah, that's good news if you were dragging in High School, you're GPA should climb a bit. But High School overachievers should beware, your GPA is most likely to dip. D'oh.

Below is my implementation of this solution. You can also find it on github. As usual I find myself preaching the benefits of Scheme. The code below was written on my Galaxy Note 5 using Termux, emacs and Tinyscheme. With relative ease I was able to implement a generator framework that works for both lists and data files. I'm also able to leverage the Scheme reader so that the data file format is trivial to operate on. Finally, I wrote a generic sigma function that walks through my data set once, but performs all the various summations I'll need to calculate the necessary values. In other words, I feel like I've got an elegant solution using little more than lists, lambda functions and sexprs. It's beautiful and should be memory efficient.

Here's the code:

;; https://programmingpraxis.com/2016/06/10/linear-regression/

(define (show . args)
  (for-each (lambda (arg)
       (display arg)
       (display " "))
     args)
  (newline))

(define (as-list x)
  (if (list? x) x (list x)))

(define (g list index)
  (list-ref list index))

(define (make-list n seed)
  (if (= n 0) '()
      (cons seed (make-list (- n 1) seed))))

(define (list->generator lst)
  (let ((remaining lst))
    (lambda ()
      (cond ((null? remaining) '())
     (else
      (let ((x (car remaining)))
        (set! remaining (cdr remaining))
        x))))))

(define (file->generator path scrubber)
  (let ((port (open-input-file path)))
    (lambda ()
      (let ((next (read port)))
        (if (eof-object? next) '() (apply scrubber next))))))

(define (sigma generator . fns)
  (define (update fns sums data)
    (let loop ((fns fns)
        (sums sums)
        (results '()))
      (cond ((null? fns) (reverse results))
     (else
      (let ((fn (car fns))
     (a  (car sums)))
        (loop (cdr fns)
       (cdr sums)
       (cons (+ a (apply fn (as-list data)))
      results)))))))

    (let loop ((data (generator))
        (sums (make-list (length fns) 0)))
    (if (null? data) sums
 (loop (generator)
       (update fns sums data)))))

;; Magic happens here:
;; m = (n × Î£xy − Σx × Î£y) ÷ (n × Î£x2 − (Σx)2)
;; b = (Σy − m × Î£x) ÷ n
(define (linear-regression data)
  (let ((sums (sigma data
        (lambda (x y) (* x y))
        (lambda (x y) x)
        (lambda (x y) y)
        (lambda (x y) (* x x))
        (lambda (x y) 1))))
    (let* ((Sxy (g sums 0))
    (Sx  (g sums 1))
    (Sy  (g sums 2))
    (Sxx (g sums 3))
    (n   (g sums 4)))
      (let* ((m (/ (- (* n Sxy) (* Sx Sy))
     (- (* n Sxx) (* Sx Sx))))
      (b (/ (- Sy (* m Sx)) n)))
 (cons m b)))))

(define (make-crystal-ball data)
  (let* ((lr (linear-regression data))
         (m  (car lr))
         (b  (cdr lr)))
    (lambda (x)
      (+ (* m x) b))))

;; Playtime
(define (test)
  (define data (list->generator '((60   3.1)
      (61   3.6)
      (62   3.8)
      (63   4.0)
      (65   4.1))))
  (let ((ball (make-crystal-ball data)))
    (show (ball 64))
    (show (ball 66))
    (show (ball 1024))))

;; From:
;;  http://onlinestatbook.com/2/regression/intro.html
;;  http://onlinestatbook.com/2/case_studies/sat.html
(define (gpa-sat-test)
  (define data (file->generator "gpa-sat-data.scm"
                                (lambda (high-gpa math-sat verb-sat comp-gpa univ-gpa)
                                  (list high-gpa univ-gpa))))
  (let ((ball (make-crystal-ball data)))
    (show "2.7 => " (ball 2.7))
    (show "3.0 => " (ball 3))
    (show "3.5 => " (ball 3.5))))

(gpa-sat-test)

Tuesday, March 15, 2016

The technology that powered geeks from Newton to Aldrin

Pop Quiz: What do Issac Newton, Buzz Aldrin and My Dad all have in common? They all solved math problems, at one time or another, using the same device: the mighty slide rule. After watching this video I realized I've never done a post on the slide rule, and as an aficionado of all things simultaneously basic and techy, I needed to correct that.

The technology really does go back hundreds of years, with Issac Newton clearly describing a device which anyone doing math in the 1960's or before would appreciate:

Mr. Newton, with the help of logarithms graduated upon scales by placing them parallel at equal distances or with the help of concentric circles graduated in the same way, finds the roots of equations. Three rules suffice for cubics, four for biquadratics. In the arrangement of these rulers, all the respective coefficients lie in the same straight line. From a point of which line, as far removed from the first rule as the graduated scales are from one another, in turn, a straight line is drawn over them, so as to agree with the conditions conforming with the nature of the equations; in one of

And here's an actual shot of Buzz Aldrin in space with a slide ruler floating in the background:

One can imagine that nearly every major undertaking involving complex math utilized the slide rule in one way or another. Consider this description of the building of the Golden Gate Bridge:

Charles Ellis and Leon Moisseiff calculated forces for the Golden Gate Bridge with only a circular slide rule and a manual adding machine. They worked for months -- by hand -- sometimes solving equations with more than thirty unknowns.

Those guys weren't messing around.

My favorite slide rule store does happen to come from the space program, specifically from the Apollo 13 mission:

Perhaps the most impressive use of the slide rule was during the Apollo 13 crisis. Engineers had to recalculate data to guide the crew safely back to Earth—and they had to do it quickly. Richard Ogle describes how it went down:

""NASA had at its disposal whole banks of sophisticated computers, of course, but these were not programmed to do the basic calculations [Commander Jim Lovell] needed. So the engineers reverted to that most mundane pre-pocket calculator workhorse, the slide rule. Had the engineers at Mission Control been forced to do the necessary calculations on paper, Apollo 13 might well have missed the trajectory needed to bring it back to earth altogether. Instead the slide rule, designed specifically to exploit human beings’ highly developed visual acuity, allowed them to perform a set of complicated calculations in seconds.""

OK, we all get that the slide rule is an amazing device. But how do you truly grok it? You could pick one up on eBay for under $10 or so and work towards mastering it. Or, you could install the App on your phone for free, and start playing with it right now. But, as they say, if you truly want to understand something, teach it. And I know of no more recalcitrant a student than the computer. So I decided to program a slide rule.

Here's my solution:

(struct scale (symbol location abs-fn scale-fn) #:prefab)

(define C (scale 'C 'slide
   (lambda (s)
     (log s))
   (lambda (a)
     (exp a))))

(define D (scale 'D 'body
   (lambda (s)
     (log s))
   (lambda (a)
     (exp a))))

(define CI (scale 'CI 'slide
    (lambda (s)
      (log (/ s (/ 1 s))))
    (lambda (a)
      (/ 1 (exp a)))))

(define DI (scale 'CI 'body
    (lambda (s)
      (log (/ s (/ 1 s))))
    (lambda (a)
      (/ 1 (exp a)))))


(define A (scale 'A 'body
   (lambda (s)
     (log (expt s (/ 1 2))))
   (lambda (a)
     (expt (exp a) 2))))

(define B (scale 'B 'slide
   (lambda (s)
     (log (expt s (/ 1 2))))
   (lambda (a)
     (expt (exp a) 2))))

(define R (scale 'R 'slide
   (lambda (s)
     (log (expt s 2)))
   (lambda (a)
     (sqrt (exp a)))))

(define K (scale 'K 'body
   (lambda (s)
     (log (expt s (/ 1 3))))
   (lambda (a)
     (expt (exp a) 3))))



(define (make-slide-rule)  
  (let ((offset 0))
    (lambda (action . args)
      (define (align s1 v1 s2 v2)
 (cond ((and (eq? (scale-location s1) 'body)
      (eq? (scale-location s2) 'slide))
        (set! offset
       (- ((scale-abs-fn s1) v1)
   ((scale-abs-fn s2) v2)))
        offset)
       ((and (eq? (scale-location s1) 'slide)
      (eq? (scale-location s2) 'body))
        (align s2 v2 s1 v1))
       (else
        (error (format "Must align body to slide, not ~a to ~a"
         (scale-location s1) (scale-location s2))))))
      (define (read s1 v1 s2)
 (cond ((and (eq? (scale-location s1) 'body)
      (eq? (scale-location s2) 'slide))
        ((scale-scale-fn s2) (- ((scale-abs-fn s1) v1) offset)))
       ((and (eq? (scale-location s1) 'slide)
      (eq? (scale-location s2) 'body))
        ((scale-scale-fn s2) (+ ((scale-abs-fn s1) v1) offset)))
       (else
        (error (format "Must read from slide to body, or body to slide. Not ~a to ~a"
         (scale-location s1) (scale-location s2))))))
      (case action
 ((align)
  (apply align args))
 ((offset) offset)
 ((read)
  (apply read args))
 (else
  (error (format "Unknown slide rule command: ~a" action)))))))

I found this page especially useful in developing my implementation.

What I learned from this little coding exercise was that there's really only two operations that you can perform with a slide rule: slide it and read it. So I implemented those two operations: align and read. For example, to perform basic multiplication, you align the C and D scales, and then read the value off the D scale relative to the C scale. Or, in code:

(define sr (make-slide-rule))
(sr 'align C 1 D 5)
(sr 'read C 3 D) ;; => 15

And here's an example of computing X3 using the K scale:

(define sr (make-slide-rule))
(sr 'align C 1 D 1)
(sr 'read C 3 K) ;; => 9

I definitely appreciate that by implementing more advanced scales, you could knock out some pretty hairy math with relative ease.

Don't get me wrong, I love me a calculator (even with its flaws) and gladly perform even the most basic mathematics on it. But from a design and hacking perspective, you really can't beat the mighty slide rule. It's definitely worth taking the time to understand how it works and why it was such an effective tool for so long.

Wednesday, January 13, 2016

Mystery Solved: One Approach to Spot-It Game Card Generation

Spot-It is one of my go-to quick games for kids and adults alike. Many a wait at the pediatrician's office has been made that much more bearable thanks to this handy little game. While there are a few variations for playing it, they all involve the same basic strategy: the faster you can spot the match between any two Spot-It cards, the better. Which brings me to the cards:

The magic of the game is this: every card matches every other card exactly once. Take a moment and confirm this by looking at the cards above.

Every time I play Spot-It I think to myself: how the heck did that do that? And so after a rousing game a few weeks ago I decided I'd have to find out. And what better way to do so than to write some code to generate my own deck of cards?

And so began a multi-week saga in puzzling out a solution. I have to admit, the problem pretty much kicked my butt. If you look at the history of my code, you'll see all sorts of wrong headed attempts. From a random statistical solution, to mutating vectors full of vectors, I tried in vain to crack the problem. For the last few weeks our table has been covered in little scraps of paper, like so:

But I lived with the problem, and during last week's dentist cleaning, I had a bit of breakthrough. Luckily, I had to wait for Shira to finish up, and so that gave me some time in the waiting room to work on a solution. I can't imagine what the receptionist thought as I tore apart pages from my pocket notepad to create prototype cards. From there, I busted out my keyboard and coded up my new insights. Alas, that solution had flaws, but it took me in the right direction.

Finally, I realized I needed a solution that had a component of backtracking to it. Luckily, before I jumped in and implemented amb, and I discovered I could get this backtracking by using simple recursion. I finally figured out the the last iteration of a solution on a jog a few days ago, and then last night I typed it in. And voila!, I give you a generated set of Spot-It cards:

OK, the above uses numbers instead of funky symbols. But the core principle holds: each card (implemented as a list of numbers) matches each other card exactly once. The prettification of the output is left as an exercise to the reader.

I'm sure there's a much more elegant solution to this problem than mine. However, for now, I'm just enjoying that feeling that comes from living with, and ultimately conquering, a problem. Whoo!

And here's the code. The mystery to Spot-It card generation (or at least, one version) is revealed below:

;;
;; Not an actual programming praxis exercise, but it should be.
;; Implement the spot-it game card sequence
;;
;; Every card matches every other card in exactly one way
;;

#lang scheme

(define (inc x)
  (+ x 1))
(define (dec x)
  (- x 1))

(define (make-symbol-generator)
  (let ([x 0])
    (lambda ()
      (set! x (inc x))
      x)))

(define (card-match? c1 c2)
  (cond ([null? c1] #f)
        ([null? c2] #f)
        ([member (car c1) c2] #t)
        (else (card-match? (cdr c1) c2))))

(define (make-deck n sym-gen)
  (if (exact? (sqrt n))
      (apply append
             (for/list ([i (sqrt n)])
               (let ([sym (sym-gen)])
                 (for/list ([j (sqrt n)])
                   (list sym)))))
      (error "Deck size must a perfect square")))
  
(define (can-add-to-pile? card pile)
  (cond ([null? pile] #t)
        ((not [card-match? (car pile) card])
         (can-add-to-pile? card (cdr pile)))
        (else #f)))

(define (grow-pile pile size deck)
  (cond ([= size 0] pile)
        ([null? deck] #f)
        (else
         (let ([card (car deck)])
           (if (and (can-add-to-pile? card pile)
                    (grow-pile (cons card pile) (dec size) (cdr deck)))
               (grow-pile (cons card pile) (dec size) (cdr deck))
               (grow-pile pile size (cdr deck)))))))


(define (remove-pile-from-deck pile deck)
  (filter (lambda (card)
            (not (memq card pile)))
          deck))

(define (match-pile pile sym-gen)
  (let ([sym (sym-gen)])
    (map (lambda (card)
           (cons sym card))
         pile)))

(define (done? deck)
  (let loop ([cards deck])
    (cond ([null? cards] #t)
          (else
           (if (andmap (lambda (c)
                         (card-match? c (car cards)))
                       deck)
               (loop (cdr cards))
               #f)))))

(define (tick deck sym-gen)
  (let ([psize (sqrt (length deck))])
    (let loop ([i 0] [deck deck] [piles '()])
      (cond ([= i psize]
             (apply append (map (lambda (p)
                                  (match-pile p sym-gen))
                                piles)))
            (else
             (let ([pile (grow-pile '() psize deck)])
               (loop (inc i)
                     (remove-pile-from-deck pile deck)
                     (cons pile piles))))))))

(define (go size)
  (let ([sym-gen (make-symbol-generator)])
    (let loop ([deck (make-deck size sym-gen)])
      (cond ([done? deck] deck)
            (else
             (loop
              (tick deck sym-gen)))))))
  

(go 9)
(go 16)

Monday, November 23, 2015

Reading Racket Documentation on Android, No Internet Needed

I've got emacs and racket running nicely on my Android phone (thanks to the awesome Gnuroot). But that left me with a new wrinkle: what's the best way to access Racket Documentation while programming on my phone? Obviously, I can open up a browser to doc.racket-lang.org but what if I'm a context where I don't have access to the Net?

By default, apt-get appears to install racket documentation in /usr/share/doc/racket, which means that my phone had the documentation, I just needed a way of viewing it.

Method 1: w3m. A quick and easy way to view the documentation is to run apt-get install w3m, and install w3m. w3m is an extremely handy tool as it lets you view html files from the command line. For example:

Method 2: launch a local web server. While w3m is a fast and functional solution, I wanted to be able to view the documentation in a standard browser, too. I figured I could accomplish this if I kicked off a local webserver. I could then point my regular Android browser to this server and I'd be all set. While I'm sure I could have installed apache or another full featured server, I found an easier and lighter weight solution: wbox. wbox is a testing tool that happens to offer the capability of being a stand alone, zero configuration, web server. Installing wbox is as easy as running apt-get install wbox. And here it is in action:

(Note the URL is: http://localhost:8081)

wbox worked perfectly, and I should be able to leverage this same approach to hosting any local documentation.

Combine this local web browser approach with Android Multi Window, and you've got a remarkably functional environment.

I'm one step closer to a full self contained, no-Internet-needed, development environment.

Thursday, November 19, 2015

Thermaroid: Android + a Thermal Printer = Super Cheap, Super Low Quality Photos on the Fly

Thermaroid lives! Today I finished smooshing together my Camera API progress with my past success printing to a Bluetooth Thermal Printer. The result is a drop dead simple app. The app starts up with absolutely no UI, it's just a live camera preview. Short press on the screen and the camera focuses. Long press the image is captured and sent to the Bluetooth printer. Here's an action shot:

(Yes, that's a screenshot of me using Thermaroid to take a picture of a picture that I just printed out with Thermaroid. Got that?)

The app needs quite a bit of polishing before it's ready for prime time, including: some indication that your photo is being printed and proper code for backgrouding the printing process. But functionality wise, it works.

I think I'm finally getting the hang of using Kawa Scheme to build apps. Apparently I needed to let go of this notion that I could somehow write Java code using Scheme's terse conventions. Kawa lets you do this a bit, but at the end of the day, things seem to work better (or compile with fewer complaints) when you annotate your code with types. I'm definitely interested in seeing if I can do some refactoring to eek out a bit more elegance than purely translating Java to Scheme. We'll see.

Check out the code here. And happy photo'ing!

Wednesday, November 18, 2015

Doing Battle With, and Losing To, the Android Camera2 API

I'm back to fiddling with with my Android Bluetooth Thermal Printer. Specifically, I'm building a small camera app that outputs to the printer directly, rather than saving files on your phone.

Looking at the Camera API, I realized that it was now deprecated, so I started building the app using Camera2. I'm not proud to admit it, but Camera2 beat me. I give up. That's one complete and crazy API to use. That's not a bad thing, it's just not ideal for quick little apps.

Oh, and all of this is written in Kawa Scheme. So if you're looking for Scheme Android examples, or how you might invoke the camera from a Scheme app, check out my project's repository, hopefully it'll be of some use.

For now, the app does little more than open up the camera, switch into preview mode and focus when you press the screen. But hopefully with the Camera functionality in place, hooking in my existing Thermal Printer Code shouldn't be too painful.

Two gotchas that I overcame:

1. I was getting the message: A TextureView or a subclass can only be used with hardware acceleration enabled, and of course the camera didn't work. I tried turning on Hardware Acceleration in the AndroidManifest.xml. That made no difference. Ultimately, it was this note that solved the issue:

The default value is "true" if you've set either minSdkVersion or targetSdkVersion to "14" or higher; otherwise, it's "false".

Turns out, I wasn't setting minSdkVersion at all. I did so (setting it to 22), and that fixed the issue.

2. I fired up the camera preview but the screen remained black. There were no messages in the Android monitor so I was baffled. Then it hit me: the camera is lying down on its lens, of course the screen is black. I picked it up, and the camera view showed the correct image. What the heck was I thinking?

Here's the project if you want to follow along: Thermaroid

Tuesday, April 07, 2015

A Little Multi-Radix Programming Fun

I was sucked in by today's Programming Praxis Challenge: Pounds, Shillings, Pence. The goal was to create a couple math functions that worked with mixed radix values (for example, time which is specified in weeks, days, hours, minutes and seconds). Developing the API turned out to be some good fun. The exercise definitely reminded me of my very early days of programming, where you'd be given an assignment to make proper change (damn you pesky quarters, nickles and dimes!) and would have to div and mod your way to a solution. I recall being thoroughly stumped by these sort of tasks, so perhaps this was my way of slaying a very old dragon?

I ended up taking a page from Unix, and built my 'system' on the notion of converting multi-radix values to a single integer value. In the time example above, I convert all values to seconds (just like Unix does) to allow for easy math.

You could hardly call what I put together elegant, but I'm pleased with the results. It almost lets you think about the values you'd like to (in the case below, miles, yards, feet and inches) versus what the computer ideally works in.

(define dist '(1760 3 12)); read: (yards-per-mile feet-per-yard inches-per-foot)

(define short    (make-mr dist '(5))          ; 5 inches
(define football (make-mr dist '(100 0 0))    ; 100 yards
(define marathon (make-mr dist '(26 385 0 0)) ; 1 marathon

(show (mr-add football marathon))
(show (mr-sub marathon football))
...

And here's the complete code:

;; Mixed Radix Math --
;; http://programmingpraxis.com/2015/04/07/pounds-shillings-pence/
;;

(define time-spec '(7 24 60 60))
(define dist-spec '(1760 3 12))
(define hms-spec '(60 60))
  
(define (mr-factors spec)
 (let loop ((spec (reverse spec)) (current 1) (result '(1)))
  (cond ((null? spec) result)
        (else
         (let ((x (* (car spec) current)))
          (loop (cdr spec) x (cons x result)))))))

(define (mr-normalize spec value)
 (cond ((= (length value) (+ (length spec) 1)) value)
       ((> (length value) (+ (length spec) 1))
        (error "Invalid value: " value spec))
       (else
        (mr-normalize spec (append (list 0) value)))))

(define (mr->int spec value)
 (let ((factors (mr-factors spec))
       (cleaned (mr-normalize spec value)))
  (apply + (map * factors cleaned))))
  
 (define (int->mr spec value)
  (let loop ((value value) (factors (mr-factors spec)) (mr '()))
   (cond ((null? factors) (reverse mr))
         (else
          (let* ((f (car factors))
                 (q (quotient value f))
                 (r (remainder value f)))
           (loop r (cdr factors) (cons q mr)))))))

(define (show . x)
 (for-each display x)
 (newline))

;;
;; Slightly higher level API
;;

(define (make-mr spec value)
 (cons spec value))

(define (mr-spec x) (car x))
 
(define (mr-value x) (cdr x))

 
(define (mr-op op)
 (lambda (x y)
   (let ((xv (mr->int (mr-spec x) (mr-value x)))
         (yv (mr->int (mr-spec y) (mr-value y))))
     (make-mr (mr-spec x) (int->mr (mr-spec x) (op xv yv))))))
   
(define mr-add (mr-op +))
(define mr-sub (mr-op -))

 
(define x (make-mr hms-spec '(3 19 45)))

Thursday, March 19, 2015

The "Funny, It Doesn't Look Like a Linux Server", Server

Check out the cutest edition to our Linux family:

What you're looking at is a TP-Link TL-WA850RE WiFi range extender. A while back, I was having WiFi woes, so I picked up this $30 WiFi extender from Amazon. Turns out, the extender didn't help matters much, so I decided to put it to use in another way.

I installed OpenWRT on the device. OpenWRT is a Linux distribution designed for routers and the like, and it caught my eye because it had confirmed support for this particular device. Installing OpenWRT was almost too easy. I grabbed the .bin file (it was in the ar71xx » generic subdirectory) and used the upload firmware option that was available in the built in web based UI.

In just a few minutes I turned this hunk of white plastic into a Linux box, which, well did nothing. Through some small miracle, I was able to hook it up to a cable and telnet to it.

The first order of business was to configure this device as a WiFi client (or station) rather than the default configuration of being an access point. My hope that was once the device was in client mode, I could plug it into the wall in a random spot in our house, it would then boot up and I'd be able to telnet/ssh to it.

I found this and this article handy is setting up client mode on the device. However, it was ultimately this bit of advice that made all the difference:

If the target network uses the 192.168.1.0/24 subnet, you must change the default LAN IP address to a different subnet, e.g. 192.168.2.1 . You can determine the assigned WAN address with the following command: ...

I had wanted to setup the lan (wired side) of the device to have a static IP and the wan (WiFi side) to have a DHCP picked up IP. It wasn't obvious, but attempting to have both the static IP and dynamic IP be on the same network caused it to fail. The static IP would be set, but the WiFi side wouldn't ever be properly configured. Here's the configuration that ended up working for me:

# /etc/config/wireless
config wifi-device 'radio0'
        option type 'mac80211'
        option channel 'auto'
        option hwmode '11g'
        option path 'platform/ar934x_wmac'
        option htmode 'HT20'
        option disable '0'

config wifi-iface
        option device 'radio0'
        option network 'wan'
        option mode 'sta'
        option ssid 'SSID_TO_CONNECT_TO_GOES_HERE'  # [1]
        option encryption 'wep'
        option key 'PASSWORD_GOES_HERE_SEE_BELOW'   # [2]


# /etc/config/network
config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config interface 'lan'
        option ifname 'eth0'
        option force_link '1'
        option proto 'static'
        option ipaddr '192.168.2.75'     # [3]
        option netmask '255.255.255.0'

# /etc/config/firewall

# ... trimmed ...
config zone
        option name             wan
        list   network          'wan'
        list   network          'wan6'
        option input            ACCEPT  # [4]
        option output           ACCEPT
        option forward          REJECT
# ... trimmed ...

Some notes from above:

[1] - This is where you specify your router's SSID to connect up with
[2] - For WEP encryption I entered a hex value here, versus text. I used this site to do the conversion.
[3] - This was key: my router will give a 192.168.1.x IP, so this needs to be off that network.
[4] - Once I got everything set up, I was getting a connection refused message when trying to telnet to the server. The wan firewall needed to be changed to allow access

Once this all got hashed out, I was able plug the device into a random spot on the wall and telnet to it. Success! And yet, where do I go from here?

Obviously this is useful for educational purposes. I've already had to brush up on my basic networking skills to get this far, and there's plenty more to learn. Heck, you could use this $30.00 router to learn about life on the command line and generally how to be a Unix geek.

OpenWRT, however, is more than just a learning platform. There's a large number of software packages available, and they can be installed using opkg with ease. Turning this bad boy into a web server or the like should be easy enough. I was even able to install a version of scheme, by grabbing an older sigscheme package:

root@pipsqueak:/# opkg install http://downloads.openwrt.org/attitude_adjustment/12.09/ar71xx/generic/packages/sigscheme_0.8.3-2_ar71xx.ipk
Downloading http://downloads.openwrt.org/attitude_adjustment/12.09/ar71xx/generic/packages/sigscheme_0.8.3-2_ar71xx.ipk.
Installing sigscheme (0.8.3-2) to root...
Configuring sigscheme.
root@pipsqueak:/# sscm 
sscm> (map (lambda (x) (* x 9)) '( 1 2 3 4 5 6))
(9 18 27 36 45 54)
sscm> (exit)

Ultimately, what will make this useful is if I can find an application for the device that leverages its near invisible profile and dirt cheap price. If I was in the security business, or a nerd-action-novel writer, then the uses would be pretty obvious. Walk in, plug in device, walk out. And bam! you've got a server that can try to worm it's way onto the network. But for myself, I'm going to have to think a little more on this. Perhaps the device should live in my car? Or maybe it'll be useful in a hotel room? Not sure, but the technology is just too cool to ignore.

Wednesday, February 25, 2015

Adventures in Dithering: Making some gray in a sea of black and white

Yesterday I implemented support for printing images in my DropPrint Android app. One issue with the printer is the range of values it prints: mainly it has none. As they say: you can have any color you want, as long as it's black. So a typical photo, which is filled with all sorts of grays, gets turned in a photo filled only with black and white. Like this one:

In some cases this effect may be desirable, but I was curious if I could leverage the halftone effect to simulate shades of gray. With a halftone you trick the eye into seeing gray by by varying the mixture of white and black dots. One Google search told me that while halftoning may get the job done, there's another important option to consider:

If you are doing this because you like the effect, that's cool. But if you just want to dither down to a black and white image consider using a "Floyd-Steinberg" dither. It provides high quality results and distributes the error across the image. http://en.wikipedia.org/wiki/Floyd-Steinberg_dithering

The Floyd–Steinberg dithering looked exactly like what I wanted, and the Wikipedia page even gave me an algorithm I could translate to scheme code:

(for-each-coord src-w src-h
                (lambda (x y)
                  (let* ((old-pixel (rgb->gray (pixels (idx x y))))
                         (new-pixel (if (> old-pixel 128) 255 0))
                         (quant-error (- old-pixel new-pixel)))
                    (store! x y new-pixel)
                    (update! (inc x) y       (* quant-error (/ 7 16)))
                    (update! (dec x) (inc y) (* quant-error (/ 3 16)))
                    (update! x       (inc y) (* quant-error (/ 5 16)))
                    (update! (inc x) (inc y) (* quant-error (/ 1 16))))))

After fixing update! so it wouldn't try to update pixels outside of the image (I'm looking at you (-1, 1)), I was surprised at the quality of the image generated. Here's the same image as above, but with dithering in place:

Look at that, there's now some "gray" in the image!

I'm not sure what to make of the white vertical bars. They are almost certainly a defect in code as I've printed other images that don't have them.

The main issue I have with this algorithm is that it's terribly slow. Dithering a 384x447 pixel image takes almost 30 seconds, with the vast majority of that time spent looping over every pixel in the image. I'm sure I'm doing something inefficient, though it's possible that I'm running into a performance issue with Kawa Scheme. At some point, I'll probably debug it further and see why it's so slow.

Next up: I've got to see if I can get rid of those annoying white bars and then I need to make the Bluetooth connectivity far more bullet proof. When that's done,I should have a pretty dang functional app.

As usual, the DropPrint source code can be found here.