I'm at it again, making images for a Google Map custom map using ImageMagick. As with last time, I'm impressed with how effortlessly these tools can make slicing and dicing large images. But, I also ran into a fresh set of issues.
Error while extracting a PNG from a PDF
convert -verbose -density 1322.6667 map.pdf map.png
kicks off the following Ghostscript command:
"gs" -q -dQUIET -dPARANOIDSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=0 \ "-sDEVICE=pnmraw" -dTextAlphaBits=4 -dGraphicsAlphaBits=4 "-r1322.67x1322.67" \ "-sOutputFile=/tmp/magick-XXtlfTcB" "-f/tmp/magick-XXmlIqok" "-f/tmp/magick-XXGNxDG3"
And for large values of density were failing with the message:
Error: /ioerror in --showpage-- Operand stack: 1 true Execution stack: %interp_exit .runexec2 --nostringval-- --nostringval-- --nostringval-- 2 %stopped_push --nostringval-- --nostringval-- --nostringval-- false 1 %stopped_push 1894 1 3 %oparray_pop 1893 1 3 %oparray_pop 1877 1 3 %oparray_pop --nostringval-- --nostringval-- 2 1 1 --nostringval-- %for_pos_int_continue --nostringval-- --nostringval-- 1777 0 9 %oparray_pop --nostringval-- --nostringval-- Dictionary stack: --dict:1158/1684(ro)(G)-- --dict:1/20(G)-- --dict:82/200(L)-- --dict:82/200(L)-- --dict:108/127(ro)(G)-- --dict:295/300(ro)(G)-- --dict:23/30(L)-- --dict:6/8(L)-- --dict:22/40(L)-- Current allocation mode is local Last OS error: 28 GPL Ghostscript 9.02: Unrecoverable error, exit code 1 convert: Postscript delegate failed `map.pdf': No such file or directory @ pdf.c/ReadPDFImage/611.
I figured I had to be running out of some resource, but I couldn't tell what. To help with that, I installed strace (sudo yum install strace) and ran the command:
strace -f -o /tmp/strace.out convert -verbose -density 1322.6667 map.pdf map.png
convert crashes like one would expect. While looking through the tail end of /tmp/strace.out I noticed this line:
24405 write(6, "\377\377...trimmed..."..., 131072) = -1 ENOSPC (No space left on device)
Of course! A df -h confirmed it: I was running out of drive space in /tmp.
The fix for this was especially easy because I'm running on Amazon AWS. In the Amazon AWS console I made a new, 50Gb EBS partition. I then mounted it from within the server. I then created a new temp directory and made use of it by setting the variable TMPDIR.
In other words, I did something like so:
sudo mount /dev/sdf /mnt/ sudo mkdir /mnt/ben sudo chown ben /mnt/ben mkdir /mnn/ben/tmp export TMPDIR=/mnt/ben/tmp convert -verbose -density 1322.6667 map.pdf map.png
Took a heck of a long time to finish, but finish the command did.
Error while creating tiles
Creating 256x256 tiles using ImageMagick is almost trivial. I just ran:
convert -verbose -crop 256x256 map.png tiles/t.png
However, for especially large map.png files, I received the message:
24318 Killed convert -verbose -crop 256x256 map.png tiles/t.png
I figured it was the system that killed off the process, and so I checked /var/log/messages to see why. Sure enough, there was this line present:
DDD XXX kernel: [35532.036652] Out of memory: Kill process 24318 (convert) score 986 or sacrifice child
Running out of memory definitely made sense. A few options to fixing this included having the system use swap space or upgrading to a larger Amazon instance. Luckily, there's an easier way. Turns out, you can tell convert that it has limits to the amount of resources that it can allocate. Sure enough, changing the above command to one that convinced ImageMagick it had less memory available to it worked like a charm.
convert -limit memory 5GB -limit map 5GB -verbose -crop 256x256 map.png tiles/t.png
The command may have taken longer to complete, but it worked just as smoothly.