I Make Things - CLI

Fix for Tor on Leopard

nospam@example.com (Latcarf) — Mon, 03 Dec 2007 04:07:44 -0400

After installing Mac OS X 10.5 (Leopard) fresh (I won't go into details ), I needed to reinstall Tor. I found the Tor expert installer didn't quite do what it was supposed to as far as creating the _tor user the daemon runs as (the user ends up a little mangled - check it out after install with dscl . -read /Users/tor) Since Leopard doesn't use NetInfo (niutil) anymore, I had to manually create the user with DirectoryServices (dscl). Here's the quick-and-dirty on what I did...

# Create Tor Group and User:
sudo dscl . -create /Groups/_tor
sudo dscl . -create /Groups/_tor PrimaryGroupID 100
sudo dscl . -create /Groups/_tor RealName "Tor Group"
sudo dscl . -create /Groups/_tor Password '*'

sudo dscl . -create /Users/_tor
sudo dscl . -create /Users/_tor UserShell /usr/bin/false
sudo dscl . -create /Users/_tor RealName "Tor Server"
sudo dscl . -create /Users/_tor UniqueID 100
sudo dscl . -create /Users/_tor PrimaryGroupID 100
sudo dscl . -create /Users/_tor NFSHomeDirectory /Library/Tor/var/lib/tor
sudo dscl . -create /Users/_tor Password '*'
sudo dscl . -append /Groups/_tor GroupMembership _tor

After that, there was still a little more to do to get Tor to run. I needed to create a place for the logs and fix the StartupItems script. Here's what I did for that...

# Create the logs directory
cd /var/log
sudo mkdir tor
sudo touch tor/tor.log
sudo chown -R _tor:_tor tor

# Fix perms on the "home" directory
cd /Library/Tor/var/lib
sudo chown -R _tor:_tor tor

# In /Library/StartupItems/Tor/Tor change TORGROUP from TORGROUP=daemon to:
TORGROUP=_tor

# Start the Tor server
sudo /Library/StartupItems/Tor/Tor start

Cool. That fixed it. Still not sure whether I want logging in /var/log/tor or /Library/Tor/var/log/tor though... Also, I guess I probably should submit all this to the Tor folks too. I'm so damn lazy.

Download all Space.com Wallpapers

nospam@example.com (Latcarf) — Mon, 09 Apr 2007 01:01:17 -0400

This is a really quick and dirty script I banged out in about a half-hour to download all the wallpapers at Space.com. I like reading the articles at Space.com from time to time and I found the wallpapers there are pretty cool. I wanted to grab them all and use them for my Mac OS X screensaver. Of course, downloading them individually (click, click, click, right-click, save as, etc... - ad infinitum), is a pain in the ass. So, the Space.com Wallpaper Download (sdcwd) script was born!

The script recursively downloads all the images of the specified size, creating folders for each collection. Just start it and let it run!

I used the following software and versions, you'll need '-o' support in grep for sure and have curl and wget installed. Of course, if the urls change at Space.com, this script will be broken. Your mileage may vary.

Software Versions:
curl - curl 7.13.1
grep - grep (GNU grep) 2.5.1
wget - GNU Wget 1.8.2

#!/bin/bash
# Space.com Wallpaper Download (sdcwd)

image_size="1280" # Set this to 640, 800, 1024, or 1280 depending on the image size you want
collections=(`curl -s http://www.space.com/php/multimedia/downloads/wallpapers/collection.php?collection=adrian_lark | \
grep -o 'option value="collection.php?collection=$.*$"' | \
sed 's,option value="collection.php?collection=$.*$",\1,'`)
collection_url="http://www.space.com/php/multimedia/downloads/wallpapers/collection.php?collection="
image_url="http://a52.g.akamaitech.net/f/52/827/1d/www.space.com/entertainment/downloads/spaceart/images/"

logFile="sdcwd.log"
topDir="Space.com_Wallpapers"

getImages()
{
this_collection="$1"
collurl="${collection_url}${this_collection}"

if [ ! -d "$this_collection" ]; then
mkdir "$this_collection"
cd "$this_collection"

# Log
echo "----------------------------" >> "../$logFile"
echo "$this_collection" >> "../$logFile"
echo "----------------------------" >> "../$logFile"

curl -s "$collurl" | grep "/php/multimedia/imagedisplay/wallpaper_display.php?pic=.*_$image_size\.jpg" | \
grep -o "pic=$.*.jpg$" | \
sed "s,pic=,$image_url," | \
tee -a "../$logFile" | \
xargs wget -q

# Log
echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~" >> "../$logFile"
echo >> "../$logFile"

cd ../
# If we didn't download any pics for this directory (maybe the requested size wasn't available)
# remove the empty directory.
find "$this_collection" -type d -empty | xargs rmdir
fi
}

old_dir="`pwd`"
mkdir "$topDir"
cd "$topDir"
for this_collection in ${collections[@]}
do
getImages "$this_collection"
done
cd "$old_dir"

Recursively Convert Line-Endings

nospam@example.com (Latcarf) — Wed, 07 Mar 2007 22:15:32 -0400

$ find . -type f -exec perl -pi -e 's/\r\n?/\n/g' {} \; ¬

Found on Madness & Macness

Convert Date to UNIX Timestamp

nospam@example.com (Latcarf) — Tue, 09 May 2006 13:39:28 -0400

I looked all over trying to find an easy way to convert a date to a UNIX timestamp in a shell or perl script. I was disappointed that I didn't find anything that was easy, sure and didn't require extra software. Most of my UNIX scripts have to operate on Mac OS X so I don't concern myself too much about portability and generally try to use what's already available on the system.

The easiest way I found to convert a date string to a UNIX timestamp is with PHP's strtotime function. Fortunately, PHP is included on Mac OS X machine at no extra charge! This one-liner handles the need pretty well.

$ my_stamp=`php -r \ ¬
'fwrite(STDOUT,strtotime("2006-05-08 15:23:52"));'` ¬
$ echo $my_stamp ¬
1147127032

The constants: STDIN, STDOUT, and STDERR are predefined to make life easy!

For more info, check out: "Using PHP from the command line."

Recursively Delete Subversion Properties

nospam@example.com (Latcarf) — Fri, 14 Apr 2006 19:08:47 -0400

My conversion from cvs to svn caused a bunch of files to end up with svn:keywords set for Author, Date, Revision, and Id. I recursively deleted the svn:keywords property from all the files with the following command. This could work with other svn property commands as well.

$ cd ~/sandbox/project/ ¬
$ find . ! -path "*svn*" \
> -exec svn propdel "svn:keywords" "{}" \; ¬

iconv Character Set Conversion

nospam@example.com (Latcarf) — Fri, 01 Apr 2005 17:57:17 -0400

I ran in to an issue the other day where I needed to perform some regular expression matching on a file and then edit the file's contents. Not a big problem really, I do it all the time, but this case was different as the file involved was encoded in UTF-16.

I see more and more Unicode files these days and at first I was worried by what I might have to do to manipulate this particular file. A little bit of research led me to something that made life super easy!

The UNIX command (available on Mac OS X and Debian boxes) 'iconv' does the work for me. 'iconv' can convert to any character set encoding available on your particular system. For my purposes, I just converted my UTF-16 file into a MacRoman file. Next, I manipulated to my heart's content, then I re-converted to UTF-16. It was easy as pie. I also used the command to convert a whole bunch of UTF-16 files to UTF-8, it was no problem. Here's the actual command in action:

$ iconv -f UTF-16 -t MacRoman my_file > my_new_file ¬

Awk's Built-in NF Variable

nospam@example.com (Latcarf) — Tue, 29 Mar 2005 15:26:58 -0400

Use the awk 'NF' (number of fields) built-in variable as an easy way to get the last field of a multi-word output. Here is an example using Mac OS X's hdiutil command to get just the calculated checksum of a disk image.

$ foo=`hdiutil checksum -type UDIF-CRC32 myimage.dmg` ¬
$ echo $foo | awk '{print $NF}' ¬
$5C63390B

Mac OS X RAM Disk

nospam@example.com (Latcarf) — Wed, 23 Mar 2005 19:13:38 -0400

Here's a cli way to create RAM disks on Mac OS X. Unless you have a huge amount of RAM, you'll probably not want to do this. Besides, Mac OS X has surperb i/o caching and virtual memory management so you might not see much of a performance boost with a RAM disk. I compiled one of our apps on a RAM disk and it didn't make a lick of difference in compile speed. Anyway, here's how to create the RAM disk:

$ NUMSECTORS=1228800 # 600MB ¬
$ VOLUMENAME=RAM_Disk ¬
$ mydev=`hdid -nomount ram://$NUMSECTORS` ¬
$ newfs_hfs -v $VOLUMENAME $mydev ¬
$ mkdir /Volumes/$VOLUMENAME ¬
$ mount -t hfs $mydev /Volumes/$VOLUMENAME ¬

Here's is how to remove the RAM disk:

$ umount $mydev ¬
$ hdiutil detach $mydev ¬
$ rmdir /Volumes/$VOLUMENAME ¬

Can't Export Arrays in BASH

nospam@example.com (Latcarf) — Mon, 14 Mar 2005 20:03:50 -0400

After fiddling for hours with a script born of a Frankenstein marriage between AppleScript and bash, I found out what I am trying to do impossible. >:-o

Arrays cannot be exported in the bash shell. Let me say that again for posterity. You cannot export an array in bash, ever! This fact is a real pain in the ass. I think I understand why; The environment is actually an array of variables and bash only supports one-dimensional arrays. Therefore, you can't move an array into an array.

Continue reading "Can't Export Arrays in BASH"

Output man pages to a file

nospam@example.com (Latcarf) — Wed, 09 Feb 2005 15:49:48 -0400

$ man "command_name" | col -b > ~/filename ¬

A good way to create user home directories in Mac OS X

nospam@example.com (Latcarf) — Tue, 21 Dec 2004 20:00:00 -0400

$ sudo cp -R /System/Library/User\ Template\ ¬
> /English.lproj/Users/myuser
$ sudo chown -R myuser:staff /Users/myuser ¬