Tuesday, October 31, 2023

DSLR vs Cell Phone Camera: To Schlep or Not to Schlep

Before each adventure I go on, whether it's a trip around the block or around the world, I have the same debate: should I lug along my Canon T6s DSLR? On one hand, the telephoto lens and old school camera form factor lets me capture pics I'd miss out on if I relied solely on my cell phone. On the other hand, the T6s is fragile, bulky, heavy and relatively dated. My cell phone, a Galaxy S22 Ultra has a remarkable camera system and its 10x telephoto lens lets me record photos that would have been impossible using previous phones.

Lingering in the back of my mind is this question: if the DSLR is such an asset, then perhaps I should upgrade? I was gifted the T6s back in 2016. In 7+ years cell phone camera technology has noticeably improved, could the same be said for the DSLR market? If so, maybe a camera refresh is in order.

To answer these questions, I want to compare my DSLR and camera phone pics from our travels over the summer. The challenge here is that I've got thousands of pictures to sort through and no easy way to compare them. 

So let's build the tools I'll need to simplify this comparison. And then let's put the DSLR to the test: is it worth its weight, or time to retire it?

Up next: normalizing exif data for easy photo comparison.

Tuesday, October 24, 2023

On the Difference Between Answers and Understanding

Both my Father and my Father-in-Law (z"l) were science teachers. Between my Dad's teaching non-majors and my FiL teaching high school, I'm certain they both strived to impart a love and appreciation of science to a skeptical audience. In their own ways, they imparted to me the fundamental difference between understanding versus simply having the right answer. My father-in-law would pour over physics textbooks the same way most people would re-read a favorite novel; all along the way relishing the fundamental. My Dad would pass along the warning he gave his students about spreadsheets: they'll always give you an answer, but is it the right answer?

Yesterday, as I fiddled around with the math behind stadiametric range finding I had a moment where this notion of understanding a topic crystalized. Typically, range finding is used to find an unknown distance. The forumula I was working with was as follows:

ad   od
-- = --
ah   oh

In every YouTube video I saw on the topic, the goal was to calculate the distance to some target, ad (actual-distance). But, it ocurred to me that you could use this same forumula and technique, yet flip around what you're solving for.

Say you were with a buddy and you wanted to run some 100 meter sprints. You could plug the known values into the formula above:

100m    66cm
----  = ----
5'6"     oh

In this case, we can solve for oh (observed-height). Doing this, comes to:

(100m / 5'6")        = (66cm / oh)  ; plug in the numbers
(10,000cm / 167.64cm) = (66cm / oh)  ; convert to cm
(66cm x 167.64cm) / 10,000cm = oh ; solve for oh
1.1cm = oh

This is what 1.1cm looks like marked out on my wallet-sized ruler:

I can now find 100 meters by walking away from my buddy and every so often holding this marked-off ruler out in front of me. When my running partner completely fills the marked off bounds of the ruler, I I've hit 100 meters. Science for the win!

While this may be a very minor bit of mathmetics, it made me smile when it all clicked for me. I'm confident both Dads would appreciate my little discovery and that I'm grasping their wisdom.

Over the years, I've watched as answers have become easier and easier to come by. Calculators, spreadsheets, Google, Wikipedia and Stack Overflow; oh how wonderful these services can be and how quickly they can reveal an answer. And now we seem to have reached peek-answer availability with Chat GPT. You can not only have the answer you want, but in your preferred voice and tone. Who could ask for more? And yet, understanding is still the gold the standard. Perhaps Shel Silverstein put it best:

The Homework Machine,
Oh, the Homework Machine,
Most perfect
contraption that's ever been seen.
Just put in your homework, then drop in a dime,
Snap on the switch, and in ten seconds' time,
Your homework comes out, quick and clean as can be.
Here it is— 'nine plus four?' and the answer is 'three.'
Three?
Oh me . . .
I guess it's not as perfect
As I thought it would be.

Monday, October 23, 2023

The $1.50, Ultra Compact, Field Expedient Rangefinder

I wanted to see if I could DIY a stadiametric rangefinder. But, before I could start crafting I needed to understand the math behind this distance estimation technique.

One clue as to how stadiametric rangefinding is calculated, comes from the unit of measure found in military spotting scopes: the mil. Mil stands for milliradian and is a measure of angular distance. From this, I inferred that I was going to be digging into trigonometry. I braced myself some complex math. When I finally untangled the how this all works, I found the process to be delightfully simple. In fact, we can trivially derive it.

Decoding the Math

Consider this crude drawing of me standing in front of an object:

The object is ad (actual-distance) away from me, and is ah (actual-height) tall. Assume that I know ah, and am trying to figure out ad. Now imagine I grab a ruler and hold it up in front of me. The picture now looks like this:

The length of my arm is od (observed-distance), and I can read off of the ruler how tall the object appears to me, oh (observed-height).

The magic, if you will, is that these measurements are in proportion. That is:

od   ad
-- = --
oh   ah

Intuitively, this makes sense: how tall the object appears to me is directly related to how close I stand to it. But, there's no reason to trust your gut here: the mathematical principle in play is similar triangles and it's more than worth your time to read up on it.

As with any proportion, if you know three values you can calculate the fourth. This suggests that there's nothing to actually build to create this type of rangefinder. I simply need a ruler I can hold out in front me. As luck would have it, the wallet sized fresnel lens I carry in my man-bag has both inch and centimeter markings on it. By the principles above, I should be able to calculate distances in the field using it. Let's try this out.

Let's Try It

I situated myself on the edge of my driveway and peered down to a bank of traffic lights in the distance. My goal: find the distance to the traffic lights.

I held up the wallet sized magnifier, and read .7 centimeters off of it. Keep in mind, all I'm interested in is the ruler on the magnifier; I'm ignoring the magnification itself.

Using a sewing style measuring tape, I measured the distance from my face to my outstretched arm. That came to 66cm. I also Googled around to figure how out how tall a bank of traffic lights may be. The consensus was that each light was 12". So I used: (12" x 3) + 4", or 40", as my estimate. I now had all the information I needed to fill into the formula above:

(ad / ah)      = (od / oh) ; the formula
(ad / 40")     = (66cm / .7cm) ; known values plugged in
(ad / 101.6cm) = (66cm / .7cm) ; inches to cm
ad             = 101.6cm x (66cm / .75cm) ; solve for 'ad'
ad             = 9,579cm ; divide and multiple values
ad             = 95.79m ; convert cm to m

According to the math above, the stop light was about 96 meters away. I then pulled up Google maps, switched to satellite view, and measured the distance on the map. It came out to a shockingly close 92 meters. I'm amazed that my estimate was so close to what I measured in Google Maps. Still, I suppose the math is sound, so I can't really complain.

A Little Field Prep

While there may be nothing to build to do this kind of range finding, there is important information to know. To make performing these calculations in the field easier, I created a text file containing some common unit conversions, as well as some sizes of standard objects (thanks range-r-card for inspiration!). Here's what in the file:

Arm length: 66cm

To centimeters
inch x 2.54
feet x 30.48
yard x 91.44
meter x 100
kilometer x 100,000
mile x 160,934

From centimeters
cm / <factor above>

Objects
traffic light: 12" (30.48cm)
person: 5'9 (175.26cm)
avg SVU: 1.9m (190cm)
door: 2m (200cm)
semi truck: 3.5m (350cm)

I printed this information out as a cheat sheet and did a bit of poor-man's laminating by covering it with packing tape. I then tucked it in to the same slot that holds the fresnel lens.

And just like that, I'm ready to do some field expedient distance estimation. And best of all, the setup was free and doesn't add any bulk to my gear.

Friday, October 20, 2023

In Search of a Low Cost and Compact Field Expedient Distance Calculator

Imagine you're a sniper (or hunter, or golfer, or surveyor), to do your job you need know the distance from yourself to some far off point. To get this value, you'd probably reach for your high tech laser, infrared or sonic rangefinder. But what if you're rangefinder's batteries are dead or the device damaged? All isn't lost, as you're probably using an optic that includes a ticked-crosshair like this:

Those dots aren't just for decoration, they are designed for estimating distance. This method is known as stadiametric range finding, and it works by having the viewer plug two values into a math formula. The values are the actual height or width of an object in the scene, and the number of ticks the object takes up in the scope. For example, if you know there's a 5'6" (1.67 meters) person in the scene, and this person takes up 3.5 ticks in your mil-dot scope, then you know that you are approximately ((1.67 x 1000) / 3.5), or 477 meters away from that person.

This method is handy because you don't need a special piece of tech to use it. The optic you are already schlepping around can have this built in. There's nothing extra to carry, break or forget.

What interested me however, was whether it's possible to use this same technique without an optic. Whether in a wilderness or urban setting, being able to estimate distances would be a handy skill. I can't, however, justify carrying around a mil-dot monocular on the off chance I'll want to measure a distance.

The good news is that you can absolutely do stadiametric range finding without magnification. The range-r-card by Black Hill Designs does exactly this. You hold the plastic card in front of you, and estimate distance using the same principle as a mil dot reticle.

While more purpose built, the Dead On Range Finder does the same thing. The Dead On is used by bow hunters, and on the surface doesn't look anything like a sniper's scope or range-r-card. However, the principle remains the same: you know an object's size in the scene (the average belly height of a deer), an observed size of this object and therefore, an approximate distance.

This type of range finding isn't new. Back in the 1800's when soldiers switched from muskets to rifles they found themselves with both an opportunity and a challenge. Rifles had a much larger range than muskets, but one could only take advantage of this if you could properly estimate the distance to your target. The December 1st, 1869 edition of the North Iowa times gave this grim prediction if the challenge of fast and accurate range finding was ever solved:

An Italian officer is reported to have made a discovery, by means of which any private soldier is enabled to measure the distance of any object within range, instantly, and at the same time to aim a gun or canon with unerring accuracy. This would make all firearms such murderous weapons, that two detachments of troops within range would be enabled to utterly destroy one another.

During the Civil War, the US issued stadia to select soldiers as a means of doing, you guessed it, primitive stadiametric range finding.

The Civil War stadia, like the range-r-card, works by holding the device at just the right distance from your eye by having the operator bite down  on, and extend a cord. When an enemy soldier fills up a marked section of the stadia, you effortlessly know his distance.

While both the range-r-card and 19th century stadia replica are relatively cheap and compact, I was curious if it was possible to DIY a solution from gear that I already carry. That way, I won't have to take up space on a purpose built item that will get used infrequently.

Stay tuned, I'm planning to tackle that challenge next.

Tuesday, October 17, 2023

Review: A Prayer for the Crown-Shy

I zipped through Becky Chamber's A Psalm for the Wild Built and was eager to read the second book in the series: A Prayer for the Crown-Shy. A Prayer turned out to also be a quick and enjoyable read. Like A Psalm I found myself arguing, sometime aloud, with the text. But for me, that's very much the joy of this series.

Below are some of these ideas. I'd consider them spoilers, so if you haven't read A Prayer for the Crown-Shy, do so now.

***Spoilers Below***

A Psalm for the Wild Built was marked by four notable features: the normalization of gender-free characters, the lack of physical conflict, a detailed description of features that power the Woke Utopia the characters inhabit, and the declaration of an unexpected organic trait by the robot population. To quickly recap, in A Psalm, the main character is sibling Dex. Not brother Dex or sister Dex. The fact that sibling has no gender is taken as a given, and plays no obvious role in the plot. Other than a few fleeting moments in A Psalm, the characters are never in any real danger. They appear to live in a crime and corruption free society. The main conflicts of the story are emotional, not physical. In A Psalm, we learn how society has finally come in to balance with nature: single use materials (like 'oil plastic'), are considered an anathema. Finally Mosscap explains, unintuitively, that the robots of the planet opt to die. They do this to emulate the organic life around them.

Does A Prayer for the Crown-Shy deliver on all four of these fronts? Yes, yes it does.

First, on the topic of gender, the text makes use of the title Mx. Because I listened to the text instead of reading it, it took a few moments to register what was going no. Mx. is pronounced 'mix', and I thought at first this might be a character's first name. Then I realized that Sibling Dex and Mosscap were talking with a stranger, and as such it would be respectful to refer to them as 'Mr.' or 'Ms.' Mx. is apparently the title to use when you don't want to specify a gender.

The use of Mx. in the text is perfect: it normalizes non-gendered interactions, yet does so in a casual non-plot impacting way. It's both an inconsequential choice (Chambers could have opted for a 'Mr.', 'Mrs.', or 'Ms.' title) and a deeply consequential one (Chamber is offering an education for folks like myself, and that's appreciated).

Next, there's the question of whether the characters are in any real danger. The short answer is, no. There's a moment involving the coastal town they visit where Sibling Dex confesses to Mosscap that the community they are approaching won't be fond of their presence. For a second, I thought Chambers might veer off course from A Psalm and put Dex and Mosscap in danger. I could imagine the scene: townsfolk descend on Dex and Mosscap armed with pitchforks and torches. Alas, it doesn't happen. The conflict with this community is purely an emotional one. The premise is that they don't like technology, and so they won't like Mosscap. As Mosscap explains, and I'm paraphrasing here, to not be liked does not feel good.

I think I see where Chambers was going with this. I can guess what it must be like to grow up in a community that rejects you. That kind of environment would be a source of significant trauma.

But, to tour the country and arrive in one town out of many where you aren't adored, and somehow feel sad, seems excessively fragile. I'm not buying it. Put another way, when I drive into southern Virginia, my self worth doesn't dip because I see Trump and Let's Go Brandon signs everywhere.

Also, I'm not convinced that the anti-technologists of the coastal town would even have a problem with Mosscap. From what we learn of them, they bristle at the way technology can lead humans down a flawed path. But Mosscap isn't human. If anything, I think they'd want to see for themselves how Dex interacts with Mosscap. Does Dex treat Mosscap like a piece of technology to be harnessed for the betterment of his life; or does he treat Mosscap as a sentient entity. They would find the latter, which I think would fit with their world view.

Next, I expected Chambers to school me on the Woke Utopia that Sibling and his fellow citizens live in and I was not disappointed. In this book we learned about 'pebs.' Pebs are a facility for tracking how much you've contributed to society. Exchanging pebs seem like a transaction: Bob gives Sally 1 peb, and Sally gives Bob a loaf of bread. But, its purely informational. Bob can always get the bread from Sally. The use of pebs just provides a public metric to let society know the current status of contribution vs. consumption.

This is a fascinating idea and is rooted in the idea that people fundamentally want to contribute. Sure, Bob could game the system by collecting up loaves of bread and sitting around playing video games all day. But if he did that, he would be showing he has a root issue he needs help with. Maybe he doesn't know how to contribute (Bob, open up a video game parlor - duh!), or maybe he's unwell (Bob recently lost his cat, and needs emotional support). What seems out of the possibility is that Bob is a moocher. They don't have moochers in Sibling Dex's world, apparently.

Pebs are interesting, but what I found especially powerful was Sibling Dex's response when Mosscap suggested, 'oh, so pebs are like money.' Dex responds the same way you or I might if you mentioned to a visitor that you have a housekeeper come on Wednesdays and they responded,  'oh, so you have slave.' You'd quickly explain that while a housekeeper and a slave may have some very shallow similarities, the analogy doesn't work. Slavery is irredeemably evil. And so it goes with money as it relates to pebs. Money is a big no-no.

Finally, there's the question of what unexpected declaration Mosscap will make about his robot clan. Here too, Chambers delivers. Mosscap explains that the robots have observed that all life eventually goes extinct. Therefore, robots take no special measures to fight off their own extinction. Eventually their parts will all fail, and robots will fade from existence, like dinosaurs or the passenger pigeon.

So robots opt to die and to let themselves go extinct. I appreciate that Chambers is looking for some controversial, yet fundamental truths about life here. And while the robots have a point, I think they've misread what nature is trying to teach them. More core than these two principles are that (a) every being clings to life, and (b) every being evolves.

Yes, every plant and animal will die. But before that, every plant and animal will fight to exist and to allow future generations to exist. And just as fundamentally, every plant and animal will adapt to the settings it finds itself in. One clever example of these principles is found in the text itself, where Dex notes the presence of trees wedged into the cliffs above the sea. These trees no doubt call for a generous amount of soil to exist and would normally thrive in a forest. But, life finds a way. Over time, the trees of the area became adapted to surviving in cracks between rocks. That's life. Yes, these trees will all die, and their species may ultimately go extinct. But the far more interesting story is what happens on the way to this.

By this logic, it's far more 'life like' to have Mosscap augment his body with organic material, then it is for him to opt to 'die' because a part failed.

A Prayer for the Crown-Shy checks the same boxes A Psalm for the Wild Built does. So yes, the storyline does have more than a little predictability to it. And yet, I still found the book to be an enjoyable read. The dialog I find myself having with characters is precious in its own right. I hope Chambers mixes things up in the future. But the topics she's playing with in these initial two books are thought provoking and I'm eager to join in on the debate.

Monday, October 16, 2023

Taming YouTube, Part 5: Command Line Searching

The last piece of my YouTube hacking journey was to implement command line searching for content. The idea is that rather than paging through results in a web browser, I can kick off and process searches at a shell prompt. As with many Unixy endeavors, the approach initially seems less powerful than an interactive UI. But, given the ability to script and post process results, I'm confident that the command line approach will ultimately win the day. 

Let's see this in action. Suppose I want to catch up on a classic outdoorsy topic: Altoids Survival Kits. The idea is to pack an Altoids candy tin with the essentials for dealing with an emergency in the woods.

Here's what my initial search looks like:

$ youtubeassist -a search -q "Altoids Survival Kit" | wc -l
     198

$ youtubeassist -a search -q "Altoids Survival Kit" | head -5
0-fj28qufo0|2023-09-23T01:35:37Z|Rivers End Outdoors|Altoids Tin Can be used for many other things. #Shorts #Bushcraft #Survival.
kqPGdBdB8ow|2023-09-20T22:04:00Z|Best Damn EDC [Taylor Martin]|I put together an EPIC EDC Urban Altoids Survival Kit 2023
LXf-fiAhOlo|2023-08-20T06:53:35Z|Ashton's EDC|My Take on the Altoids Tin: What I Keep in Two Different Altoids EDC Kits
dpwEhyV-VbA|2023-08-18T18:15:04Z|Jon Gadget|My 2023 Altoids EDC Essentials Kit (Everyday Carry)
cECC341Y074|2023-08-10T00:35:18Z|Typical Chris|How to Make A Mini Survival Kit Out of an Altoids Tin!

My search turned up only 198 hits out of the bazillion of these videos on YouTube. That's because I limit my search results to the first 200 hits.

If I wanted to move all 200 of these videos to a playlist for easy watching I could say:

# make the playlist
$ youtubeassist -a playlist-create -n "Search Hits"
PLBuxEPZOHHKZE-uDfDReNTN6bmUu3r_WE|private|Search Hits

# -o 'output's the results to the requested playlist
$ youtubeassist -a search -q "Altoids Survival Kit" -o 'Search Hits'
...bye bye quota...

Outputting to a playlist helps bridge the divide between the command line and interactive worlds.

It's also possible to process the output and add videos to a playlist after the fact. For example:

# Put all fire related videos into a 'Fire Making' playlist
$ youtubeassist -a search -q 'Altoids Survival Kit' -F | \
    grep -i fire | cut -d'|' -f1 | \
    youtubeassist -a playlist-items-add -n "Fire Making"

The -F in this case shows the 'full results', not just the most recently executed search.

Because executing a search is quota heavy, I've set it up so that searches are cached for one hour. After an hour, re-running a search will show you only the new results. For example:

# wait a couple of hours ...
 
$ youtubeassist -a search -q "Altoids Survival Kit" 
EHK820jd32c|2023-07-09T22:58:59Z|Mount Kephart Outdoors|Altoids tin Kits USLESS? Blackie Thomas does it again Bushcraft & Wilderness Survival Gear
PA_TpbwCfQQ|2023-07-01T17:00:22Z|The Modern Rogue|Filling an Altoids Tin with crime tools �
jXWqPw27Yk8|2023-06-28T20:00:10Z|Survival Dispatch|From the City to the Wilderness: EDC Survival Kit | Fuel The Fires
dIuYFZ1zRN4|2023-05-02T21:21:13Z|WayPoint Survival|Perfect Belt Survival Kit?  [ It has ALL the stuff! ]
eHuMgL4Ex-k|2023-03-03T00:12:05Z|Comandos Benin Prepper|EDC melhor kit de sobrevivência do mundo. Altoids é passado?
DbnHhFhG2Bo|2023-02-21T01:27:17Z|WayPoint Survival|Old School Pocket Survival Kit!
WCb6kVgMGl0|2023-01-28T16:02:48Z|David Hindin, M.D.|Squeezing an operating room inside an Altoids tin
mZiasleIwxM|2022-10-09T22:58:26Z|Buckeye Bushcraft|Ultralight Survival Kit For Hunters
u5wieAfgR3o|2022-03-12T16:23:17Z|Freddy Ramos|Custom Sheath Bushcraft Utility Pouch for Altoids Tin
4zrjx40_gHc|2021-09-29T22:34:09Z|Grim Granite|Is survival gear REALLY possible from a DOLLAR STORE?
8X4dHYnPdUM|2021-04-09T09:00:03Z|Ultimate Survival Tips|IT WORKS! Cheap DIY Survival Kit Compass Hack - Altoids Tin Micro Compass Kit...
sGyv_dkWLfk|2020-10-14T00:24:13Z|JoeDom|Altoids Survival Tin
h6dQK4tp2yI|2020-10-09T17:43:28Z|Lone wolf Survival|Altoid Tin Survival Kit
IFF1vmMnSjo|2020-10-03T05:28:05Z|Adventures with Cowboy Col|20 Altoids Survival Tin/Kit Items In A Wallet ? The ULTIMATE EDC WALLET!
YgOvhC0_lCw|2020-10-01T01:58:59Z|Overlander Benny|DIY altoid survival kit tin! More than your average tin. Fire, water, hunt, fish from your kit!
jO3il9sdLG8|2020-09-15T15:13:12Z|Rockos Modern Survival|Altoids survival kit
LmfaNcvYS60|2020-09-09T19:49:15Z|Swagger|Altoids Survival Kit
QIJr7RuYgoc|2020-08-19T00:09:08Z|Old Time Skills|How to make an Altoids tin survival kit
UrKMGooOYZc|2019-10-05T17:01:34Z|yamato623|The altoids smalls survival kit $15
$ youtubeassist -a search -q "Altoids Survival Kit"  | wc -l
      19

$ youtubeassist -a search -q "Altoids Survival Kit"  -o "Search Hits"

By default, searches have their sort order set to date, which should only show me newly added content to YouTube. But, as you can see above, the results aren't that strict. I'm seeing 19 'new' hits, even though many of them are from years ago.

In general, YouTube API searches tend to be broad in what they return. For example, not all of the hits above mention 'Altoids' in their title. Generally, this works for me as I'm often looking for more esoteric content rather than the same high-profile videos YouTube show again and again.   I can always narrow down searches by using grep and other tools.

I'm not entirely sure how I'm going to fold this search capability into my YouTube workflow. However, I am confident that running and grep'ing through command line searches will be far more efficient than mindlessly scrolling.

Give all this a try by grabbing youtubeassist from github. Happy hacking!

Tuesday, October 10, 2023

Taming YouTube, Part 4: Playlist Aging and Rotating

After much setup and cleanup, I was finally ready to code my way to a better YouTube experience. The goal I'm after is to craft a workflow that has me spend an accountable amount of time queuing and watching videos, and helps avoid clutter and brain overload.

The workflow I envisioned consists of a series of playlists and rules for moving videos betwee these lists. Here's what I'm planning:

  • Incoming - where videos are queued up for watching.
  • In Progress - where longer videos live after I've started watching and they have caught my interest.
  • Stale - where videos are moved to after they've spent too long on Incoming or In Progress.
  • Nice - where videos that were educational, inspirational or hilarious end up.
  • Wow - where life changing videos end up.

The idea is that once a day I queue up videos in Incoming. This limits my zombie scrolling time on YouTube to a single session. From there, it's fair game to watch Incoming and In Progress videos throughout the day. Running youtubeassist -a playlists-rotate moves videos either between or off playlists. The rules are tersely defined in $HOME/.config/youtubeassist/config:

ROTATE_DIR=$HOME/dt/i2x/youtube-archive/src/main/playlists/archived
ROTATE_RULES="Incoming:7:Stale: \
      Stale:14::Stale \
      In_Progress:14:Stale: \
      Nice:7::Nice \
      Wow:30::Wow"

The format is: playlist, time-to-live in days, destination playlist followed by destination directory. For example, videos stay on the Incoming playlist for 7 days, and then are moved to Stale. Videos live on the Stale playlist for 14 days and then are moved to the Stale directory, found inside of the $ROTATE_DIR. This takes them off of YouTube, stashing them in a git repository for long term archival.

The Nice and Wow playlists are ultimately archived in a git repo, too. My expectation is that I'll eventually mine these directories, publishing a list of my favorite videos over a given time or subject.

Here's the output of me rotating playlists for today:

$ youtubeassist -a playlists-rotate
name=Incoming, ttl=7, output=Stale
name=Stale, ttl=14, dir=/Users/ben/dt/i2x/youtube-archive/src/main/playlists/archived/Stale
name=In Progress, ttl=14, output=Stale
name=Nice, ttl=7, dir=/Users/ben/dt/i2x/youtube-archive/src/main/playlists/archived/Nice
name=Wow, ttl=30, dir=/Users/ben/dt/i2x/youtube-archive/src/main/playlists/archived/Wow

Implementing -a playlists-rotate was easy once I'd implemented a slightly lower level operation, -a playlist-age. Aging a playlist works by taking in a number of days and moving all videos off the list that meet that threshold. Below is an example of aging items out of the Sketching playlist.

# All videos on the list for more than 15 days are moved to the 'Stale' playlist  
$ youtubeassist -a playlist-age -n "Sketching" -t 15 -o "Stale" -p preview
Aging 'Adriaen van de Velde (1636-1672) A collection of paintings 4K Ultra HD.mp4', 20 > 15
Aging 'Sketch People Loosely like a Pro: Urban Sketching Tips', 20 > 15
Aging 'How to do urban sketching', 20 > 15
...

# See, all the videos are gone
$ youtubeassist -a playlist-age -n "Sketching" -t 15 -o "Stale" -p process

# And, they were moved to the Stale playlist
$ youtubeassist -a playlist -n "Stale"
UExCdXhFUFpPSEhLYVhuUHpzdTRqUUxUMUhQb3lic29Cby4zMDg5MkQ5MEVDMEM1NTg2|czCqXPURvK0|0|9 Drawing Exercises to Improve Your Urban Sketching Skills
UExCdXhFUFpPSEhLYVhuUHpzdTRqUUxUMUhQb3lic29Cby45ODRDNTg0QjA4NkFBNkQy|05Ucm95fpTs|0|Travel Journal Sketching #1 Start up tips
UExCdXhFUFpPSEhLYVhuUHpzdTRqUUxUMUhQb3lic29Cby5EMEEwRUY5M0RDRTU3NDJC|emCpEE223RI|0|How to Draw and Sketch with a Fountain Pen - The Very Basics - Tutorial and Tips
...

# Let's see what's left
$ youtubeassist -a playlist -n Sketching
UExCdXhFUFpPSEhLYjRNZ2hwdHRnV3FNd3gzMnZjQVRsVS4zRjM0MkVCRTg0MkYyQTM0|2gY51vW2H-U|12|How to Draw Anything - 7 Easy Tips for Beginners
UExCdXhFUFpPSEhLYjRNZ2hwdHRnV3FNd3gzMnZjQVRsVS5DNzE1RjZEMUZCMjA0RDBB|BjSJCMeH_8I|11|A Brief Guide to Perspective // Urban Sketching for Beginners
...

# I can use a time-to-live of -1 days to move all items to a a playlist. In this case
# I'm moving all items to 'Incoming'
$ youtubeassist -a playlist-age -n "Sketching" -t -1 -o Incoming -p preview
Aging 'How to Draw Anything - 7 Easy Tips for Beginners', 12 > -1
Aging 'A Brief Guide to Perspective // Urban Sketching for Beginners', 11 > -1
...

# And poof, I can delete 'Sketching'
$ youtubeassist -a playlist -n Sketching

I've set up youtubeassist -a playlists-rotate to run daily from cron. Now I just need to be disciplined about keeping my mindlessly scrolling of YouTube to a once a day occurrence. That habit will take practice, but I'm confident I'll get there.

Next up: the last piece of the YouTube optimization puzzle: streamlining video discovery.

Tuesday, October 03, 2023

Taming YouTube, Part 3: Declaring YouTube Playlist Bankruptcy

I've got my youtubeassist shell script authenticated with the Google API so I can finally get down to the business of optimizing my YouTube workflow.

First things first, I wanted to clean up the 300+ playlists I currently have in my account. To do this, I added a playlists-snapshot command to youtubeassist. This action grabs all the playlists on my account, creates a fresh local directory with today's timestamp, and writes out each playlist as its own .json file. Here this option in action:

$ youtubeassist -a playlists-snapshot -d ./snapshots
[wait for it...and done]
$ ls
20231003.0709
$ ls 20231003.0709 | wc -l
316
$ ls 20231003.0709 | head
15.json
17.json
18.json
21_Years.json
A_Trip_To_The_Zoo.json
Audio_Magic.json
Bedtime_Tunes.json
Best_of_Brad.json
Blog_it.json
Carry.json

In a single command I've captured all 316 of my playlists to a local directory. I can confirm the .json files are valid by using jq.

$ cat 20231003.0709/Weekly_Discoveries_2019_02_03.json | jq -r '.items[] |  "https://youtube.com/watch?v=" + .contentDetails.videoId + "|" + .snippet.title'
https://youtube.com/watch?v=HNAM2EVXH9A|Owl City - Not All Heroes Wear Capes (Acoustic)
https://youtube.com/watch?v=I-QfPUz1es8|Imagine Dragons - Bad Liar
https://youtube.com/watch?v=jrlMaRdM4wI|LÉON - You And I (Official Video)

To complete the snapshot procedure, I set up the local directory to be part of a git repository. Once the snapshot command finished, I did a git add, git commit and git push to ensure that the snapshot was safely stored in the cloud.

With all 316 playlists captured, I'm ready to remove the majority of them. To accomplish this, I captured a list of playlists in a .txt file. I edited this file, removing any playlist I wanted save. Finally, I deleted the rest using this one-liner:

$ youtubeassist -a playlists > all.txt
$ emacsclient all.txt      # edit all.txt, removing playlists to save
$ cat all.txt | while read line ; do \
  id=$(echo $line | cut -d: -f1); \
  name=$(echo $line | cut -d: -f2) ; \
  echo $name ; \
  youtubeassist -a playlist-delete -i $id ; \
done | tee yt.cleanup
Carry
Skills
To build
Weekly Discoveries
Weekly Discoveries - 2022-12-25
...
The request cannot be completed because you have exceeded your

All went well until I realized I'd maxed out on my YouTube quota for the day:

Whoops. Part way through the list, YouTube stopped honoring my requests to delete playlists. My dreams of having a pristine set of YouTube playlists are going to have to wait for another day (or two). My quota should free up tomorrow, and I can delete the rest of playlists then. When the playlists are cleaned up, I can move to the next part of my plan: creating a workflow that let's me efficiently find, watch and archive YouTube content. Stay tuned!