Gotcha: Emacs on Mac OS: Too Many Files Open

Generally, the MacPorts version of Emacs works great on my Mac Mini. But every so often, I'd hit a Too many open files error.

The *Messages* buffer was little help as it just repeated what I already knew:

insert-directory: Opening process input file: Too many open files, /dev/null [2 times]

I'd attempt to close buffers or shut down projectile projects, but there was nothing I could reliably do to recover from this error. Ultimately, I had to do the unthinkable and restart emacs.

I tried the obvious fix: telling my Mac to allow processes to open more files. I followed this recipe:

$ sudo cat /Library/LaunchDaemons/limit.maxfiles.plist
<?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
          "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
    <dict>
      <key>Label</key>
      <string>limit.maxfiles</string>
      <key>ProgramArguments</key>
      <array>
        <string>launchctl</string>
        <string>limit</string>
        <string>maxfiles</string>
        <string>64000</string>
        <string>20480</string>
      </array>
      <key>RunAtLoad</key>
      <true/>
      <key>ServiceIPC</key>
      <false/>
    </dict>
  </plist>

$ sysctl kern.maxfiles
kern.maxfiles: 20480
$ sysctl kern.maxfilesperproc
kern.maxfilesperproc: 64000

It didn't help. Perhaps I was setting the limit incorrectly for my particular version of the OS. Or maybe I was setting the value too high, and the system was reverting it to something smaller. Or maybe all was good at the OS level, and it was a bash ulimit problem.

I considered all of these scenarios, but no matter how I set the file limit or what I set the file limit too, emacs kept hitting a 1024 open files limit. I could easily confirm this with lsof:

$ ps auxww|grep Emacs
ben              15944   0.0  0.7 412688032 111280   ??  S    Mon06AM  71:07.20 /Applications/MacPorts/Emacs.app/Contents/MacOS/Emacs
$ lsof -p 15944 | wc -l
    1618

After much frustration and searching, I finally stumbled on this reddit thread where a fellow emacs user complained about the 1024 max file limit:

Been trying to figure out a way to get out of this trap… I got a new Mac as a work laptop and I can’t seem to update the file descriptors. I updated it for the system but whenever eMacs is opened, ulimit is still at 1024.

Thankfully, there was a helpful reply:

I hit this all the time, as I work on a large monorepo with lsp-mode (and sometimes treemacs, which also watches stuff).

Whenever it happens I run M-x file-notify-rm-all-watches and things go back to normal for a while.

Last time I looked into this, you could not work around it by adjusting ulimits or literally anything. It's a core limitation of a low-level API used internally and is not configurable. I will try to find the previous discussion.

Aha! This made sense, as I've added lsp-mode to my workflow recently.

I was delighted to find I wasn't the only one having this problem, and more importantly, there was an easy work around.

Alas, when I tried to execute M-x file-notify-rm-all-watches I found that my version of emacs didn't have this function.

A quick Google search turned up this source file for filenotify.el. It does have file-notify-rm-all-watches defined as follows:

(defun file-notify-rm-all-watches ()
  "Remove all existing file notification watches from Emacs."
  (interactive)
  (maphash
   (lambda (key _value)
     (file-notify-rm-watch key))
   file-notify-descriptors))

I copied the code, untouched, into my emacs configuration. Next time I got the dreaded Too many open files error, I ran M-x file-notify-rm-all-watches and just like that, emacs was happy again. And so was I.

Surprisingly Simple: From Cassette Tape to mp3

A few weeks back I re-discovered a few mix tapes that Shira had made for me back when we were dating. While mix tapes were definitely a thing while we were growing up, I was never into music enough to think to create my own.

I do, however, have a dim memory of the neighborhood kids sitting around with a tape record placed next to a radio. When the song came on that we wanted to add to our musical collection, someone would quickly hit the record button and we'd be good to go. It was Napster before there was Napster. In this dim memory, I recall hearing Fast Car, by Tracy Chapman being over-played on the radio (yeah, they did that back then, too). Wikipedia tells me this song was released in April of 1988, when I would have been 12 - so that sort of all checks out.

Anyway, I had these now precious cassette tapes in front of me and I realized that our anniversary was fast approaching. I thought man, wouldn't it be brilliant to digitize these bad boys for the big day.

My first thought was: how on earth can I ship them to a conversion company in the short period of time I have? Then I quickly remembered, uh, this is a cassette tape, I can probably pick one up at Goodwill for less than $10. And then I realized I was still overthinking this: I bet in my ancient hardware pile I probably have a cassette player.

Sure enough, I did:

I even had the right audio cable to connect the output of the player into the microphone of the computer.

In fact, the only thing I was missing to get this setup going were C batteries. So off to CVS I went to buy a 4 of them.

I popped in the batteries and slid in one of the tapes. I hit the rewind button and the machine whirred to life. I then waited. And waited. Good lord, how long does it take to rewind a tape? With a loud click, the machine announced it was done. I hit play, and the tunes of over 19 years ago breathed to life.

I then plugged in the tape player to the computer. I was on my Mac so it took a few minutes to learn how to record audio. The short answer being: launch QuickTime Player; go to File » New Audio Recording. I then hit record on the computer and play on the cassette player. And again, waited.

When I was done, I had a 71 meg m4v file containing 30 minutes of music. I ran ffmpeg -i side_a.m4v side_a.mp3 and I was left with an mp3 file.

The whole procedure couldn't have been simpler. I can't get over how tactically satisfying it is to use a real tape player. Everything feels so, well, real. Given how easy it was to record this content, I'm going to have bust this old school tech out for my nieces and nephew and see what they think of it. Will they get a kick out of the limitations of life on cassette, or will they roll their eyes and ask to use YouTube?

Today's our anniversary and we celebrated early by listening some of the recordings last night. Mission accomplished.

Happy 19th babe!

Foot Powered macOS Monitor Hopping

We interrupt this emacs hack for an OS X hack.

Yesterday, after making progress with the Trinket Pro, I grabbed the foot pedal, wire strippers and athletic tape of all things, and connected everything together:

Yes, this is a pathetic build. No, I don't care. I loaded the Trinket with this small program:

#include <cmdline_defs.h>
#include <ProTrinketKeyboard.h>
#include <ProTrinketKeyboardC.h>
#include <usbconfig.h>


const int PIN_SWITCH = 0;    // Trinket pin to connect to switch 
const int LED        = 13;
int   pressed        = 0;
void setup()
{
  pinMode(PIN_SWITCH, INPUT_PULLUP);
  TrinketKeyboard.begin();  // initialize keyboard library
}

void loop()
{
  TrinketKeyboard.poll();
  if (digitalRead(PIN_SWITCH) == LOW)  // If the foot switch grounds the pin
  {
    if(pressed == 0) {
      TrinketKeyboard.pressKey(0, KEYCODE_F8);
      TrinketKeyboard.pressKey(0, 0);  // release key
      pressed = 1;
    }
  } else {
    pressed = 0;
  }
}

Note the use of pressed variable. It's needed to keep the board from sending a stream of key presses when its held down.

With the program above in place, pressing the foot pedal is equivalent to pressing F8 on the keyboard.

My plan was (and is) to wire this into emacs in an intelligent way. Also, I need to find a way to have the pedal respect the current modifier keys. That is, if I'm holding down shift, then it should send Shift + F8. But all that aside, once I had working pedal, I knew which annoyance I wanted to fix first.

OS X's support for multiple desktops is quite solid. If a screen has focus, it's trivial to use key combinations to navigate among desktops. What doesn't have an obvious solution is jumping the focus between screens. This is the sort of thing I found requests for, but not an obvious solution. Here's one I rigged up.

First off, I grabbed Keyboard Maestro, a piece of software mentioned in my search for a solution. Keyboard Maestro looks quite sophisticated and getting my bearings was non trivial. From poking around I learned two important pieces of information. First, the variable %CurrentMouse% contains the coordinates of where the mouse is. Second, when I'm on the 'left' screen, the x-coordinate is negative. Using these facts, I was able to rig up this rule:

By detecting the presence of the minus sign, I can tell which screen I'm on and warp the mouse to the other monitor. Finally, this is all connected to F8, which is sent by the foot pedal.

I can't tell you how gratifying this solution is. Now, when I'm on the right screen, I no longer need to remove my hands from the home row, but can just tap my foot to warp myself to the left screen. And I'm only a foot press away from being back to the right screen. It's so cool.

Gotcha of the Day: iOS Cordova App running in iSimulator Hangs on Splashscreen

I recently submitted Cordova iOS to iTunes, only to have it rejected because the app was hanging on startup. Turns out, the app worked fine on a real device, but on a simulator, the app's splashscreen wouldn't ever be hidden.

On the simulator, I did notice that if I returned 'home' and relaunched the app, it did function. It was all very confusing.

I was already working around this quirk in the splashscreen plugin:

The config.xml file's AutoHideSplashScreen setting must be false. To delay hiding the splash screen for two seconds, add a timer such as the following in the deviceready event handler

After further debugging, I learned that:

 navigator.splashscreen.hide();

was being called, yet, it wasn't having any effect.

To add insult to injury, the IO Simulator's console log wasn't showing the output of JavaScript's console.log(...) messages. The 'debugging' above involved using old school alert statements.

I solved the mystery of the missing console.log statements by following the instructions here. I'd totally forgotten that you can remotely monitor a web instance from within Safari. Frankly, it's sweet functionality, even if a bit non-intuitive.

1. On your real iDevice or in iOS simulator go to Settings > Safari > Advanced and turn on Web Inspector.
2. Desktop Safari: Safari > Preferences > Advanced and select the Show develop menu in menu bar checkbox.
3. Now that you have either iOS Simulator open or your iDevice connected to your mac start using web inspector on your website: On your mac, open Safari and go to Develop

Along with my new found console.log statements, I saw the following message being repeatedly blurted out:

Refused to load gap://ready because it appears in neither the child-src directive nor the default-src directive of the Content Security Policy.

Eager to fix *something*, even if it's not the splashscreen bug, I Googled for this message. I found this discussion, which as the message itself suggests, shows that the Content-Security-Policy meta tag needs to be updated.

To shut the app up, I went ahead and updated my CSP to be:

<meta http-equiv="Content-Security-Policy" 
   content="default-src * 'self'; style-src * 'self' 'unsafe-inline'; script-src * 'self' 'unsafe-inline' 'unsafe-eval'">

I relaunched the app, and not only did the message disappear, but the call to navigator.splashscreen.hide() worked!

And of course, this all makes sense now: the call to hide the splashscreen was failing due to a lack of permission by the page. I added the permissions and things were back to working.

Why this was failing in the simulator and not a device, I can't really say. But I'm just glad it's working.

Gotcha of the Day: Generating and Using Custom Firefox Profile Icons on OS X

One feature I rely on are Firefox Profiles. Much of my life is spent in a browser, and profiles allow me to isolate work, play and development from each other. I can be logged into my work gmail in one profile, personal gmail in another profile, and be constantly clearing my cookies in a third.

With a little tweaking, the script I used to kick off different profiles on Linux was coaxed into working on OS X:

#!/bin/bash

case `uname` in
  Darwin)
    PROFILE_DIR="$HOME/Library/Application Support/Firefox/Profiles"
    FIREFOX=/Applications/Firefox.app/Contents/MacOS/firefox
    ;;
  *)
    PROFILE_DIR=$HOME/.mozilla/firefox
    FIREFOX=FIREFOX
    ;;
esac

##
## Kick off firefox in a specific profile
##
profile=$1 ; shift
dir=`(cd "$PROFILE_DIR" ; ls | grep -e "[.]$profile$")`

if [ -n "$dir" ] ; then
  $FIREFOX -new-instance -profile "$PROFILE_DIR/$dir"  2> /dev/null &
else
  echo Profile not found: [$profile]
fi

This solution has worked well, except for one detail: in the OS X dock all the instances had the same Firefox icon. The result is that to jump to a specific profile meant a game of guess-the-browser, often jumping me to the wrong instance. It was time to clean up this icon ambiguity.

Both MultiFox and this blog post present OS X solutions for running multiple instances of Firefox. Ultimately however, I decided to stick with my shell script above and use this plugin: OS X Dock Icon Changer to customize the Firfox icon in use.

With the Dock Icon Changer extension in place all that was left to do was to pick new icons for each of my profiles: Surf, Personal, Dev and DevJr. I'm sure I could have found some graphics on Google Images or the like, but for my purposes it was easier to use ImageMagick to generate these icons. And so here's a script to do just that:

#!/bin/bash

profiles='
 Surf:BigCaslonM:#26532B:#5ABCB9
 Dev:Herculanum:#826251:#F4ECD6
 DevJr:Chalkduster:#114B5F:#F3E9D2
 Personal:Krungthep:#101D42:#5ABCB9
'

for p in $profiles ; do
  name=`echo $p | cut -d: -f1`
  font=`echo $p | cut -d: -f2`
  bg=`echo $p | cut -d: -f3`
  fg=`echo $p | cut -d: -f4`

  echo $name
  convert -background "$bg" -fill "$fg" -font "$font" -size 512x512 \
          -gravity Center label:"$name\nFF" icons/$name.png

done

Each icon has its own font, foreground color and background color. You can run convert -list font to see all the fonts available. Here's the icons the script generated:

And here's the new Dock:

And just like that, the days of playing guess the browser instance are behind me. Whoo!

Gotcha of the Day: inserting the current timestamp on OS X

I'm settling into week #2 of life on a Mac and I decided it was time to address a feature I was sorely missing: the ability to insert the current timestamp via a simple keystroke. Consider this use-case: within ChangeLog's I'll include the timestamp a patch was released, and while typing 20 or so characters is hardly a deal breaker, it's clearly annoying to have hand enter the current date and time.

Surely OS-X had a simple solution for this challenge.

Initially, I was psyched to find this automator solution to the very same problem, but I never could get it to work. Finally, at the suggestion of this post I installed and configured TextExpander to do the job.

For the 10 minutes I've had this solution in place, I'm liking it. For one thing, TextExpander let's me write shell scripts, which means that I don't need to learn the macro language that comes with it. And for another, the system is text based rather than keyboard short-cut based. So I need only to remember to type 'TS' and the current date will appear, versus trying to recall if I need to hold down Windows + Alt + Shift + T to do the same job.

I don't yet have a sense of how TextExpander fits into the larger Mac ecosystem. Is it a utility I can trust and should pay for? Not quite sure yet. Sure, I miss my autohotkey, but I'm going to try to roll with this for now.

Week one of being an Apple Fanboy | Choosing a Mac as a Primary Work Computer

OK, I'm not quite sure I'd call myself an Apple Fanboy just yet (as I'm typing this post on a Microsoft Ergonomic Keyboard). However, I have phased an OS X Mac-Mini into my work environment. Recall that since 2009 I've had a simple strategy for hardware risk management: I have two work computers, and every Monday and I switch the one in use for the week. The result is that if I have a hardware failure, I can effortlessly switch to the backup.

For years, this meant running two Windows Laptops. Then I switched the arrangement to a Windows Laptop paired with a Linux Laptop. Then it was a Linux Laptop paired with a compact Windows Desktop. And now it's official: for 2017 I plan to use a Mac-Mini paired with a similar footprint Windows Desktop Machine. Here's the new setup:

Attentive readers will note that this isn't my first time owning a Mac-Mini. I've had one on the side now for about 5 years, but I ran it just when I needed to do something OS X specific, which wasn't all that often.

So it's been essentially a week in Apple land, and so far, the biggest news is that there is no news. Most things Just Work. My core tools: emacs, bash, subversion, Firefox, etc. all work just as seamlessly on OS X as they did in Windows and Linux. It's mind blowing that I can take a totally new computer and OS, use subversion to pull down my configuration and within a few minutes be working away like this was a well-worn environment.

Consider this win: I've got a relatively fancy script I use to setup ssh tunneling. I checked it out of my subversion tools repository and ran it. And just like that, it worked. Of course, it's just standard bash commands, but still, to have something relatively complex like this not require changes is just too cool.

There's really not much to report on the hardware side. Yes, the Mac-Mini is sexier than the boxy Windows computer next to it. And at at $650 or so bucks, it's relatively cheap. But the Windows box is fan-less, so it's absolutely silent, whereas the Mac-Mini makes these tiny tinkling noises. Hardly a deal breaker, but it is noticeable. The Windows box also has far more USB ports, some of which are conveniently placed on the front panel. But still, these are minor points and from a hardware perspective the Mini gets a passing grade.

On the other hand, OS X as an operating system definitely shows promise. It has that mainstream vibe that Windows has: so tools like Skype work in a polished way out of the box. *And*, it also has a strong geek vibe: I don't need to layer Unix tools on top of the OS via Cygwin, instead they are native. At one point I struggled to get ssh-agent working properly, only to realize that it was my hacky Windows setup that was breaking things. Mac apparently supports ssh-agent out of the box and I just needed to move all that crap out of the way and let it just be.

I've warmed to a number of OS X design choices quite quickly. For example, both Windows and Mac support multiple desktops, but they have one noticeable difference with respect to how multiple monitors are handled. On Windows, when I 'slide to the desktop on the left' both monitors change. On Mac, only the current monitor changes. This allows for mixing and matching of desktops. At first this was jarring, but I think it's a win.

The biggest source of pain for the last week has been the mental grinding of gears needed to adapt to new keyboard shortcuts. Consider this keyboard conundrum: on Windows, Control + a selects all the text, and the Home key takes you to the beginning of the line. On Mac, Control + a takes you to the beginning of the line, whereas Home takes you to the start of the text field you're in. So for the last week, I've been hitting Home to get back to the start of the line, and then get annoyed because I've totally lost my place and typing at the start of the document. I've tried some recipes for overriding this behavior, but ultimately, I'm just going to need to retrain my brain.

The irony is not lost on me that years ago, when I switched from Linux to Windows, I had to stop using Control + a to get to the start of a line, only to now have to relearn this habit.

Fortunately, the keybindings in emacs are universal. So for most of my coding tasks, the keyboard short-cut changes haven't been all that big an issue.

I'd consider this first week of being a Mac Guy a solid success. My brain is going to be pissed when it finds itself on a Windows environment next week, but it'll deal. For now, I'm just happy that I can be following Rule #1 for business computing hardware: Diversify

Any killer Mac utilities or features I should know about? I need an autohotkey replacement, which I think will be AppleScript. But maybe you've got a better idea; I'd love to hear it!