Zachary Tatlock

Peter Lee and DARPA

2010-02-06T23:39:00.001-08:00

Interesting talk by Peter Lee @ University of Washington about the new DARPA and his role in its leadership.

Peter Lee is the only speaker I've seen use the phrase "cosmic beauty" in a talk (last year @ UCSD).

Putting Reviews Into Perspective

2010-02-02T11:44:00.000-08:00

This entertaining article shows how even legendary papers could get horrible reviews (html).

H.264 and HTML5

2010-01-25T16:30:00.000-08:00

Christopher Blizzard (one of the Mozilla guys) has a great post about the consequences of H.264 becoming the de facto video codec for the web. Specifically, he's referring to Google's beta of YouTube in HTML5, which is very exciting. You should opt in today! (But push for open codecs tomorrow!)

The relevant analogies are (1) the GIF fiasco and (2) MP3 patent issues.

Jack Conte Anti-VideoSong

2010-01-20T21:39:00.000-08:00

Jack Conte never ceases to cleverly entertain:

He's 1/2 of the excellent Pomplamoose duo.

TeX month name

2010-01-15T13:41:00.001-08:00

While making my CV, I needed the name of the current month. I found the trick here. Basically, you just have to create a switch:


    \newcommand{\monthname}{%
    \ifcase\number\month
    \or January%
    \or February%
    \or March%
    \or April%
    \or May%
    \or June%
    \or July%
    \or August%
    \or September%
    \or October%
    \or November%
    \or December%
    \fi}

After you've added this to the top of your TeX source, you can get the month name with \monthname.

Start svnserve at boot in Ubuntu

2010-01-12T16:13:00.000-08:00

This post provides a very simple init script to get svnserve going at boot. In Slackware, I only had to make a script executable for it to run at boot; here you must use some update-rc.d magic:

$ echo "svnserve -d --foreground -r /home/svn" \
| sudo tee /etc/init.d/svnserve
$ sudo chmod +x /etc/svnserve
$ sudo update-rc.d svnserve defaults

Fun in MN #3

2009-12-29T06:40:00.000-08:00

We recently saw the 2009 British Television Advertising Awards, a 1.5 hour movie comprised of short advertisements. Two particularly interesting types of ad:

Change blindness:
The winning Transport for London ads advocate awareness of cyclists:

Do the Test:

Whodunnit?

Both of these ads exploit "change blindness", a phenomenon where most people tend to miss drastic visual changes:

Today, Schneier noted how this could have drastic security implications. Neat stuff.
Anne and I both noticed that many of the ads were not selling anything per se. Instead they were advocating a change in behavior. For example, one ad went into gruesome detail about knife wounds and then encouraged young people to stop carrying knives. Another examined the damage caused by gun violence. Others considered substance abuse and excessive drinking. All of them used the form of an ad to do something else: promote social change. In the US, the Ad Council works similarly.

Finally the show was presented in the The Walker Art Museum, which is a fun, gorgeous place to stroll through, lots of interesting stuff. If you're in MN, don't miss it!

Fun in MN #2

2009-12-24T19:21:00.000-08:00

CA is great, but you don't get 18 inches of perfect snow in San Diego:

Here's the rest of the sledding fun.

Time Management

2009-12-24T19:13:00.000-08:00

A couple of great ideas about how to manage time (your most valuable resource!):

Time Management post from a financial advice blog.

Time Management and the Last Lecture by Randy Pausch.

Fun in MN #1

2009-12-21T19:24:00.000-08:00

The Minneapolis Sculpture Garden is a fun place, even 15 degrees below freezing:

Although it looks nicer in the summer:

Interesting Caml Flames

2009-12-21T07:09:00.000-08:00

The caml list has an interesting thread about multicore (continued).

Someone posted lack of good multicore support as a bug. Their argument is that because all machines now come with multiple cores, this is truly a bug that must be fixed. The followups:

Discuss whether this is really a bug.
Consider how one could implement better support in a mature and broadly deployed system like caml.
Give a cursory overview of the state of parallelization for functional langs (F# seems to be the favored ML for exploiting concurrency).

Jon Harrop and Yaron Minsky (both knowledgeable caml guys) weigh in at multiple points.

Holiday Decorating

2009-12-17T10:32:00.000-08:00

Anne loves the holidays. Unfortunately, in Southern California we don't get the same great holiday spirit that the Midwest fosters with its biting cold and massive snow drifts. To make matters worse, our tiny apartment can't really fit a Christmas tree. So, we started hacking and:

Of course, we spend a lot of time in the lab too, so we wanted a little holiday spirit at school as well. What's more festive than an 8 foot tall lambda in lights?

Happy Holidays!

Regifting Robin: Cute Number Puzzle

2009-12-09T08:20:00.000-08:00

Check out Regifting Robin, a neat puzzle shared by Rajesh Gupta on the misc list. Hint: if you count the number of possible answers, you'll figure out the trick.

Using bash variables in cases

2009-12-08T15:43:00.000-08:00

Pattern matching in bash via the case construct is quite useful. However, sometimes it's nice to compute the pattern you want to match against and store it as a variable. For example, the first thing one may try:

    # BAD
    PAT='"hello"|"hi"'
    
    case $FOO in
      $PAT)
        echo "matched a greeting"
        ;;
    
      *)
        echo "nothing matched"
        ;;
    esac

Unfortunately this is not the same as:

    case $FOO in
      "hello"|"hi")
        echo "matched a greeting"
        ;;
    
      *)
        echo "nothing matched"
        ;;
    esac

The first example will only match a literal "hello"|"hi", while the second would match hello or hi as we expect.

To fix this, we can wrap an eval around the entire case, which will expand all the variables before the pattern matching is carried out, thus providing the desired behavior:

    # GOOD
    PAT='"hello"|"hi"'
    
    eval "
    case $FOO in
      $PAT)
        echo "matched a greeting"
        ;;
    
      *)
        echo "nothing matched"
        ;;
    esac
    "

New Webpage

2009-12-08T15:37:00.001-08:00

I redid http://cs.ucsd.edu/~ztatlock. Feedback is appreciated.

Pomplamoose

2009-11-24T21:44:00.001-08:00

I've been watching Pomplamoose for a bit now (thanks Anne!). I really enjoy their sound and DIY ethic. They're sort of a calmer, more complex Matt & Kim (another one of my favorite bands -- thanks Chris!).

Examples include Hail Mary:

and Pas Encore (the only song I know with the phrase "internet forums"):

most

2009-11-24T19:14:00.000-08:00

The most pager does the right thing for colors without any futzing in .Xdefaults or setting multiple environment variables in .bashrc. Aside from being a drop-in replacement for more/less, the documentation mentions some other nice features, e.g. simple key remapping and "fold" like features for viewing code.


  $ sudo apt-get install most
  $ echo "export PAGER=most" >> ~/.bashrc

update:

most lacks -F, which could be a deal breaker. From less's man page:

  -F 
    Causes less to automatically exit if the entire
    file can be displayed on the first screen.

Fortunately, it looks like I can add -F support in a few lines. Unfortunately, the most source mixes tabs and spaces. UGH! Worse, indent doesn't seem to fix things automagically.

coqwc

2009-11-22T10:34:00.000-08:00

Friday we needed to measure our Coq development for the PLDI submission. Normally, I just use wc to roughly estimate program size, and

  $ grep ";\|}" FOO | wc

for a more accurate account (of C/Java code FOO). However, in Coq code one is usually interested in knowing both the amount of code you've written AND the size of the proofs about your code. Obviously, wc can't tell the difference, and so I was reduced to hacking out some grep/sed/awk monstrosity that roughly estimates the number of lines in each category.

Then today (too late) I find coqwc which does exactly what I want:

$ coqwc PRTL.v EE.v SideCond_DNM.v 
     spec    proof comments
      279        0       19 PRTL.v
      266      163       84 EE.v
       81       42       18 SideCond_DNM.v
      626      205      121 total

Brain Plasticity

2009-10-06T17:51:00.000-07:00

Gary Cottrell gave a neat talk yesterday about his lab's computational models for vision. One comment that really stuck: the brain contains many similar structures. These structure apply similar algorithms and pass their output to each other.

It seems like redundancy and self similarity are big wins for us. This also supports the idea of Neuroplasticity which roughly says that the brain can reorganize itself to handle new environments. For example, some guys at MIT wired ferret retinal outputs into the auditory cortex. The ferrets learned how to see just fine. Also, people can see with their tongues. Those examples and more in this great Discover article. Thanks Mike.

Carl Sagan + Auto Tune

2009-09-29T11:24:00.000-07:00

I cannot stop watching:
http://www.youtube.com/watch?v=zSgiXGELjbc

Install Chromium in Ubuntu 9.04

2009-09-25T10:00:00.001-07:00

Chromium is great. To use it in Ubuntu 9.04, run as root:


  # F="http://ppa.launchpad.net/chromium-daily/ppa/ubuntu"
  # echo "deb $F jaunty main" >> /etc/apt/sources.list
  # echo "deb-src $F jaunty main" >> /etc/apt/sources.list
  # KS="--keyserver keyserver.ubuntu.com"
  # K="0xfbef0d696de1c72ba5a835fe5a9bf3bb4e5e17b5"
  # apt-key adv --recv-keys $KS $K 
  # apt-get update
  # apt-get install chromium-browser

Keeping a Connection at Peet's

2009-09-03T11:31:00.001-07:00

Peet's wireless for non-http traffic is extremely bursty, to the degree that ssh and other non-http connections get starved to death. However, as long as there's enough http traffic, it seems like all connections get along OK. So, to keep ssh alive, just keep requesting junk pages in the background:


    #!/usr/bin/env bash
    
    function mksearch {
      echo -n "http://www.google.com/search?"
      echo "sourceid=chrome&ie=UTF-8&q=$1"
    }
    
    while true; do 
      for s in taco zip foo bar baz burrito cat; do
        wget --user-agent="Firefox" -O /dev/null "$(mksearch $s)"
        sleep 15
      done
    done

iPod: disable hfs+ journaling for Linux

2009-08-31T21:19:00.000-07:00

Turns out that the Linux kernel can't handle journaling in hfs+ filesystems. You can recognize the problem from this dmesg error:


    hfs: write access to a journaled filesystem is not supported,
         use the force option at your own risk, mounting read-only.

This matters if you want to write to an iPod. The best fix I've found is to disable journaling: simply mount the iPod in OSX and run:


    $ diskutil disableJournal /Volumes/iPod

where "iPod" is the name of your iPod. Once the journal is disabled, using an iPod in Linux is a cinch, particularly with tools like gtkpod or rhythmbox.

Garry Kasparov: Best Politician Ever?

2009-08-27T10:13:00.000-07:00

Kasparov is sharp in an interview: Kasparov vs. Maher.

Convert ogg to mp3

2009-08-25T21:43:00.000-07:00

Converting from one lossy format to another is a bad idea. Unfortunately, most portable music players do not support ogg. So, if you really have turn your oggs in to mp3s, you can use this:


  $ ffmpeg -i in.ogg out.mp3

[found in an ubuntu list archive]
A while ago I wrote the following "anything to mp3" script for similar tasks:

    #!/usr/bin/env bash
    
    function ext {
      # if there is a file extension, print it
      # otherwise do nothing
      echo $1 | awk 'BEGIN {FS="."} {if(NF > 1) {print $NF}}'
    }
    
    function convert {
      # convert $1 to mp3
      src="$1"
      dst="$(dirname "$src")/$(basename "$src" .$(ext "$src")).mp3"
      wav="$$.wav"
    
      # decode to wav with mplayer
      mplayer \
        -vo null -vc dummy \
        -af resample=44100 \
        -ao pcm:waveheader:file="$wav" \
        "$src"
    
      # encode to mp3 with lame
      lame -m s "$wav" -o "$dst"
    
      # get rid of temporary file
      rm "$wav"
    }
    
    # handle spaces in file names
    IFS=$(echo -en "\n\b")
    
    for src in $* ; do
      echo -n "$src ... "
      convert "$src" &> /dev/null
      echo "done"
    done

I don't know if the ffmpeg approach is better or worse.

No More Jetlag

2009-08-04T09:57:00.000-07:00

Fast 12 - 16 hours before crossing time zones, says science.

The iPhone's New Clothes

2009-08-02T17:05:00.000-07:00

New iPhone specially designed for Apple enthusiasts.

Logic in Java

2009-04-21T07:42:00.000-07:00

From Wadler's Blog, What does logic have to do with Java? (wait for the gimmick at the very end):

Remove Unwanted Icons (Windows)

2009-04-16T23:21:00.000-07:00

In Windows, there are icons that cannot be deleted by right clicking, dragging to the recycle bin, or hitting the delete key. To get rid of these pesky, unwanted icons:

Right click on the Desktop and select Properties.
Select the Desktop tab.
Click the Customize Desktop button.
Click the Clean Desktop Now button.
Check the checkboxes for any icons you do not want.
Click the Apply button.
Now the icons you despise are in a folder on the Desktop. Delete that folder.

I found these instructions here.

xdotool

2009-04-14T17:23:00.000-07:00

xdotool is an awesome command that enables programmatic control of X events, e.g. pressing keys, moving or clicking the mouse, resizing windows, etc. It is a great response to this xkcd:

You can even make the mouse infinitely loop:

    #!/usr/bin/env bash
    
    OFF=300
    RAD=300
    
    C=$(expr $OFF + $RAD)
    
    function x {
      echo "$RAD * c($1)" \
        | bc -l \
        | awk '{print int($1 + 0.5)}' \
        | xargs expr $C +
    }
    
    function y {
      echo "$RAD * s($1)" \
        | bc -l \
        | awk '{print int($1 + 0.5)}' \
        | xargs expr $C +
    }
    
    t=0
    d=0.02
    while true; do
      xdotool mousemove $(x $t) $(y $t)
      d=$(echo "$d * 1.001" | bc -l)
      t=$(echo "$t + $d" | bc -l)
    done

Watch this for a few minutes; it eventually starts doing some crazy things. The script also contains several tricks for rounding, etc. that could be more generally useful.

Twitter on Scala

2009-04-12T09:57:00.000-07:00

Here is a nice interview with Twitter developers about replacing an enterprise Ruby backend with Scala. Interesting topics include: flexibility, correctness, concurrency, and fun programming.

Best excerpt:

I think it may just be a property of large systems in dynamic languages, that eventually you end up rewriting your own type system, and you sort of do it badly.

Media Subversion

2009-04-05T21:38:00.000-07:00

Peace, Propaganda, and the Promised Land is a fairly well-made documentary available for free download. A first approximation of its theme could be:

    Coverage of the Israeli occupation of Palestine from a
    perspective absent in US news sources.

but the real point of the film is that:

    US news organizations have been subverted.

Evidence of journalistic infidelity is provided by comparing US and British reports on identical events. The differences are disturbing. Of course, this comparison alone only shows someone got it wrong, not necessarily the US. The documentary also covers history, motivation, and influence for the players involved in both the conflict and its media coverage. These factors suggest that the BBC's version of events is more reliable.

The film provides very interesting examples of how the media's language has been controlled; the effects are dramatic. The whole situation seems quite Orwellian.

Take-home point: Supplement your news intake with sources from outside your home country.

OCaml Documentation in Vim

2009-04-04T15:18:00.000-07:00

To have vim open a firefox tab to the documentation for a standard OCaml module:

Place this script :

    #!/bin/bash
    
    firefox http://caml.inria.fr/pub/docs/manual-ocaml/libref/${1}.html

in ~/bin/ocaml_lkup_module.

$ chmod +x ~/bin/ocaml_lkup_module

Add this line :

    " lookup modules in OCaml online documentation
    au BufRead,BufNewFile *.ml set keywordprg=~/bin/ocaml_lkup_module

to your ~/.vimrc.

To use, simply:

Open up an OCaml source in vim.
In command mode, move the cursor onto a standard library module name.
Press 'K'.
Tah dah! The documentation for that module is open in a new firefox tab.

Similar coverage of vim's keywordprog in a past thesaurus in vim post and from a post over at Daily Vim : Making K Useful.

Dynamically Resive Font in URxvt

2009-04-02T20:06:00.000-07:00

This script allows you to dynamically change the URxvt font size:

    #!/usr/bin/env bash
    
    printf '\33]50;%s%d\007' "xft:Monospace:pixelsize=" $1

So to change to a 16 pixel font:

    $ fontsize 16

I have no idea what the control characters mean or what they will do to other terminal emulators. I found this script on a very helpful Arch Linux forum.

Convert Anything to MP3

2009-04-02T19:53:00.000-07:00

Occasionally I need ogg files to be mp3 files. A couple days later I need wma files to be mp3s. etc. etc. This script converts about anything to mp3:

    #!/usr/bin/env bash
    
    # yield the file extension or nothing if there is no extension
    function ext {
      echo $1 | awk 'BEGIN { FS="." } { if(NF > 1) {print $NF} }'
    }
    
    # convert $1 to mp3
    function convert {
      src="$1"
      dst="$(dirname "$src")/$(basename "$src" .$(ext "$src")).mp3"
    
      # decode to wav with mplayer
      mplayer -vo null -vc dummy -af resample=44100 -ao pcm:waveheader "$src"
    
      # encode to mp3 with lame
      lame -m s audiodump.wav -o "$dst"
    
      # get rid of temporary file
      rm audiodump.wav
    }
    
    # handle spaces in file names
    IFS=$(echo -en "\n\b")
    
    for src in $* ; do
      echo -n "$src ... "
      convert "$src" &> /dev/null
      echo "done"
    done

Rip CD to wav

2009-04-01T08:54:00.000-07:00

Sound Juicer does a nice job ripping and encoding audio CDs. However, being graphical, it is not clear how to use it in a script. CDDA Paranoia fills this need; to rip all the tracks of a CD to wav files simply:

    $ cdparanoia -B

SSH Keys

2009-03-31T08:18:00.000-07:00

This guide on SSH key setup has a lot of good detail (another). Here is just the essence:

Generate and install public key:

$ ssh-keygen -f ~/.ssh/id_dsa -t dsa
Enter a passphrase and hit [Enter]. Repeat.

Install your fresh key:

    $ cat ~/.ssh/id_dsa.pub \
        | ssh MACHINE "cat - >> ~/.ssh/authorized_keys"

Now you can ssh to MACHINE without entering your password, but you have to enter your passphrase every time you want to access your private key. ssh-agent fixes this. Simply edit your .xsession so that ssh-agent starts when you login:

    change the line:
        xmonad
    to:
        ssh-agent xmonad

But replace xmonad with whatever window manager you use. Now the first time during a session you want to use your private key, you should:

    $ ssh-add ~/.ssh/id_dsa

and from then on you will be able to ssh to MACHINE without entering your password.

To get the same setup with another machine M, simply:

    $ cat ~/.ssh/id_dsa.pub \
        | ssh M "cat - >> ~/.ssh/authorized_keys"

Mechanical Marble Adder

2009-03-30T23:25:00.000-07:00

Here's a really neat mechanical adder from a brilliant woodworker:

I wonder if this could be used for some interesting lessons in K-12.

Kinetic Art by Arthur Ganson

2009-03-22T18:57:00.000-07:00

The art of Arthur Ganson is extremely appealing. Here are just two examples:

I found this on Matt Kehrt's journal, one of the interesting people met at ETAPS 2009.

The Internet is Different in the UK

2009-03-22T18:33:00.000-07:00

Turns out your geographic location changes the internet (e.g. Netflix, Hulu, YouTube, etc.). To fix simply:

    $ ssh -D 8080 -f -C -q -N LOGIN@HOST

Where LOGIN is you and HOST is running sshd in the country you want the internet to think you are from. Basically, this just makes all traffic from your machine on port 8080 go through HOST.

Now to get firefox sorted:

If you use Vimperator, get the menu back up for a bit by running:
```
    :set guioptions+=mT
```
Navigate Edit > Preferences > Network > Settings
Select Manual proxy configuration
For SOCKS Host: enter localhost
For Port: enter 8080
Make sure your settings look like this:
Click OK.
If you use Vimperator, get rid of those pesky menus with:
```
    :set guioptions-=mT
```

You are now set to go! I found this at Paul Stamatiou's site.

Steve Bourne on Bourne Shell

2009-03-19T22:15:00.000-07:00

Computerworld has a piece with Steve Bourne on Bourne shell. The most interesting bit was how he summed up the important design constraint of unquoted strings:

As the Unix command line interpreter, for example, you wouldn’t want to be typing commands and have all the strings quoted like you would in C, because most things you type are simply uninterpreted strings. You don’t want to type ls directory and have the directory name in string quotes because that would be such a royal pain. Also, spaces are used to separate arguments to commands. The basic design is driven from there and that determines how you represent strings in the language, which is as un-interpreted text.

Another interesting consideration is the importance of the shell as both an interactive environment and a scripting language. I've heard of people using toplevels in Python or even OCaml as command shells. I'm skeptical as it seems like such schemes would make common tasks noticeably more painful. What will shells be like in the future?

Deep Typechecking and Refactoring Poster

2009-03-19T11:26:00.000-07:00

At the UCSD Jacobs School of Engineering research poster session I presented our work on Deep Typechecking and Refactoring:

Caml Trading

2009-03-14T07:09:00.000-07:00

Caml Trading is a nice talk given by Yaron Minksy at CMU. The presentation briefly explains trading, motivates the use of ML in a general purpose setting, and considers areas for improvement. The Q&A at the end is particularly satisfying.

I especially appreciated this wisdom:

        You can't pay people enough to carefully review dull code.

Hooray FP!

Equality Saturation on LtU

2009-03-03T08:24:00.000-08:00

Our work from POPL 2009 on Equality Saturation is on Lambda the Ultimate, the hottest PL blog in the universe!

Coq Syntax Highlighting In Vim

2009-02-26T09:47:00.000-08:00

To get coq syntax highlighting in vim:

Download coq.vim.
Move the downloaded coq.vim to ~/.vim/syntax/

Add these lines to your ~/.vimrc file:

    " syntax highlighting for coq
    au BufRead,BufNewFile *.v set filetype=coq

Here's the official vim coq syntax highlighting page.

Make Subversion Ignore Files

2009-02-25T20:51:00.000-08:00

To make svn ignore files matching GLOB in directory DIR, first list the globs you want ignored, one per line, in some file FILE (e.g. .ignore), and then run:

    $ svn propset svn:ignore -F FILE DIR

To apply this policy recursively on all the subdirectories, add -R to the mix:

    $ svn propset svn:ignore -F FILE -R DIR

For example, if you want svn to ignore any file in the current directory matching matching *.o, first edit .ignore to contain:

*.o

and then run

    $ svn propset svn:ignore -F .ignore .

If instead, you want to ignore all object files in this directory and all subdirectories, run:

    $ svn propset svn:ignore -F .ignore -R .

Bash Loop Over Filenames With Spaces

2009-02-25T10:03:00.000-08:00

It is useful to pass a script a list of filenames and have it apply some function to each of those files. Unfortunately filenames with spaces won't work correctly in bash for loops. Each space separated part of the filename will be treated as a separate argument. (No, putting the args in quotes won't help here.)

To fix this, you simply need to change the IFS variable (internal field separator) so that it doesn't include the space character. In order to restore default behavior, you may want to save the old IFS value and reset IFS after your loop. Thus, the solution looks something like:

    OLDIFS=$IFS

    IFS=$(echo -en "\n\b")

    for f in $FILES; do
        foo $f
    done

    IFS=$OLDIFS

Here's an example of good and bad for loops:

I originally found how to handle spaces in bash loops on the very useful nixCraft site.

banner

2009-02-15T17:59:00.000-08:00

banner is useful when you need to, oh say, celebrate 1234567890 day at your department's social hour. Basically, it just generates nice, page size banners. Use is simple:

    $ banner 'your message here' | lpr -P printer

Using -w changes the size of the letters. So if you only want them to be 80 chars wide:

    $ banner -w80 'your message here' | lpr -P printer

banner manpage

Configuring urxvt

2009-02-14T07:19:00.000-08:00

rxvt-unicode is a great terminal emulator. It blows the doors off of gnome-terminal and the like in terms of performance. Commands producing significant output actually run faster. However, configuring urxvt takes a little bit more than a few clicks. In particular, you should use a .Xresources file to specify the default options you want urxvt to run with. Here's the relevant bit of my .Xresources:

    ! urxvt settings
    URxvt.background   : #202020
    URxvt.foreground   : #999999
    URxvt.font         : -*-*-*-*-*-*-18-*-*-*-*-*-*-*
    URxvt.scrollBar    : False
    URxvt.pointerBlank : true
    
    URxvt.perl-ext-common : default,matcher
    URxvt.urlLauncher     : /usr/bin/firefox
    URxvt.matcher.button  : 1 
    
    ! black
    URxvt.color0  : #202020
    URxvt.color8  : #202020
    ! red
    URxvt.color1  : #bb4444
    URxvt.color9  : #bb4444
    ! green
    URxvt.color2  : #44bb44
    URxvt.color10 : #44bb44
    ! yellow
    URxvt.color3  : #bbbb44
    URxvt.color11 : #bbbb44
    ! blue
    URxvt.color4  : #4444aa
    URxvt.color12 : #4444aa
    ! magenta
    URxvt.color5  : #bb44bb
    URxvt.color13 : #bb44bb
    ! cyan
    URxvt.color6  : #44bbbb
    URxvt.color14 : #44bbbb
    ! white
    URxvt.color7  : #999999
    URxvt.color15 : #999999

This scheme is low contrast and large font to be easy on the eyes. Crucially, the bit:

    URxvt.perl-ext-common : default,matcher
    URxvt.urlLauncher     : /usr/bin/firefox
    URxvt.matcher.button  : 1

allows you to open links from the terminal in a new firefox tab by simply left clicking on them (from here and here). This was one feature I sorely missed from the more bloated terminals. Once you've updated your .Xresources file, you'll need to run xrdb so the changes can take effect:

    $ xrdb ~/.Xresources

after which all new urxvt instances will use your settings. To make sure these options are loaded every time, you should add the above command to your .xsession file.

Wget Recursive Downloading

2009-02-08T20:01:00.000-08:00

GNU Wget is normally used from the shell to fetch a single webpage. It also has the very useful ability to fetch a directory subtree from the web by recursively following links to pages in that domain. So to get http://foo.com/bar.html and all the pages under http://foo.com that bar.html links to, simply run:

    $ wget -r http://foo.com/bar.html

To get globbing like behavior, a couple of flags do the right thing:

    $ wget -r -l1 --no-parent -A.gif http://www.server.com/dir/

-r -l1 means to retrieve recursively, with maximum depth of 1.
--no-parent means that references to the parent directory are ignored.
-A.gif means to download only the GIF files. (-A "*.gif" would have worked too.)

Base Conversion Script

2009-02-07T07:45:00.000-08:00

Here's a simple script for (somewhat) arbitrary base conversions:

    #!/usr/bin/env bash
    
    if [ $# -lt 3 ] ; then
      echo "Usage: $0 <input base> <output base> <num> ... <num>"
      exit 1
    fi
    
    b0=$1
    b1=$2
    shift 2
    
    # put semis between the remaining args
    ns=$(echo $* | sed 's/ /; /g')
    
    # convert b1 from base 10 to base b0
    b1=$(echo "ibase=10; obase=$b0; $b1" | bc)
    
    # perform conversion
    echo "ibase=$b0; obase=$b1; $ns" | bc

Note (1) the use of sed to insert semicolons between the arguments to be converted and (2) the conversion of of the output base to the input base before the conversion of the arguments. These keep bc happy and doing the right thing.

Example use:

Caveats:

Bases less than 2 and greater than 16 appear broken in bc.
Any digits in an input number that are >= the base are treated as the max digit in that base. e.g. in base 2 '123' is treated as '111'.
Hexadecimal digits need to be upper case, but this can easily be fixed by first piping the input through awk, e.g. preprocess ns with:
```
    ns=$(echo $ns | awk '{ print toupper($0) }')
```

Latex Landscape

2009-02-05T08:34:00.000-08:00

To get an entire document in landscape mode, include this line in the head of the document:

    \usepackage[landscape]{geometry}

Instead, to get only a specific page in landscape mode, first include the lscape package in the header:

    \usepackage{lscape}

and then wrap the page you want to be in landscape mode:

    \begin{landscape}
    ...
    \end{landscape}

The generated pdf will have this page in landscape mode, but the page will still be in the portrait orientation. If you want the page in the pdf to be in landscape orientation, then use the package pdflscape instead of lscape.

Landscape mode is good for long pipeline diagrams. This was found in the Blog on Latex Matters.

Wordle

2009-01-31T18:05:00.000-08:00

Wordle is a fun service that creates word collages based on an RSS feed or some other source of text. Here's the example for this blog:

I initially saw this on Wadler's blog.

Formal Certification of a Compiler Back-end

2009-01-28T06:57:00.000-08:00

Formal Certification of a Compiler Back-end (or: Programming a Compiler with a Proof Assistant) describes Xavier Leroy's technique for building a compiler that is formally guaranteed to be semantically transparent. In other words, if this compiler generates assembly program A for some input program P, then A and P are guaranteed to be equivalent. The approach centers around using the Coq proof assistant to both program the compiler and reason about the compiler's implementation. The paper is very well written and enjoyable to read.

In the first two sections the problem of compiler correctness is well motivated and a high level overview of certified, certifying, and verified compilation is provided. The author unifies these three approaches to compiler correctness and shows how they can safely be composed. The third section describes the intermediate representations (IRs) used in the compiler, which helps the reader understand the compiler's high level structure:

    Cminor    The C-like source language
              Attempts to capture subset of C most useful to embedded developers

    RTL       Register Transfer Language
              Functions are represented as CFGs with an unlimited set of registers
              Nodes in the CFG are instructions, not basic blocks
              Optimizations are performed on this IR

    LTL       Location Transfer Language
              Nodes in the CFG changed from instructions to basic blocks
              Pseudo registers changed to machine registers or stack slots

    Linear    The CFG is flattened to a list of instructions 
                  with explicit labels and branches
              
    Mach      Stack slots are mapped to actual memory locations

    PowerPC   The output language
              PowerPC chips are widely used in embedded systems
              Some pseudo-instructions (macros) are used

At the time of writing, the compiler supported constant propagation and common subexpression elimination optimizations. The operational semantics of each IR are fully formalized, and example rules are provided in the paper.

The fourth section provides details on the compiler passes and their correctness proofs. Intuitively, the proofs work by showing that if a program P0 in IR0 is equivalent to program P1 in IR1 and P0 evaluates down to P0' and P1 evaluates down to P1', then P0' and P1' are equivalent. The paper uses simulation diagrams to help make this idea clear:

                  P0   ==========   P1

                  |                 |
                  |                 |
                  v                 v

                  P0'  ==========   P1'

where the = signs signify a correspondence and the arrows show evaluation.

When everything is said and done, the compiler weighs in at roughly 36 KLOC that took "one man year" to develop (I use scare quotes because one year of Xavier Leroy hacking is probably worth significantly more that "one man year"). The quality of generated code is very good as is compile time. Next, the paper provides a nice overview of lessons learned during the development process. Some highlights include the success of Coq as a programming language and the relative difficulty of certified transformation (prove transformer correct, once-and-for-all) vs. certified verification (untrusted transformations are checked by correct verifier). Interestingly, the author suggests that certified verification should have been employed more frequently.

The final sections compare related work (where Sorin gets a shout out), and conclude. Great paper. Well worth the read.

Equality Saturation Talk

2009-01-27T15:25:00.000-08:00

Ross Tate recorded his talk from our POPL 2009 paper, Equality Saturation: a new Approach to Optimization:

It is a great talk, and weighing in at thirteen minutes, it is well worth your time to check out. If the flash version above doesn't show up, try this mp4 version. A more thorough description of this work is available on Ross's Equality Saturation page.

BTW, doing a screencast of your talk after the conference is a GREAT idea.

PLDI 2009 Accept

2009-01-27T13:18:00.000-08:00

Our work on proving compiler optimizations correct once-and-for-all got into PLDI! The other two papers from our group got in too. Hooray UCSD progsys.

Snoring Joke

2009-01-25T11:04:00.000-08:00

Ross: So Zach's snoring was soft for a while. Then it gradually got REALLY loud, like I could hear it over my headphones loud. Then it died back down till it stopped. And that repeated.

Mike: So it was kind of periodic?

Zach: Yeah, I have a sinus-oidal problem.

mutt URL Color

2009-01-22T06:48:00.000-08:00

Add this to your .muttrc to highlight URLs:

    # colors
    color body magenta black "(http|https|ftp)://[^ ]+" # picks up URLs

Lifted from a more complete mutt colors specification.

Podracer : A GNU/Linux Podcast Aggregator

2009-01-19T09:56:00.000-08:00

Podracer plus cron seems like a nice tool to keep all your podcasts up to date without relying on a pesky X app. Set up is simple:

    $ sudo apt-get install podracer
    $ mkdir ~/.podracer
    $ touch ~/.podracer/podracer.conf
    $ touch ~/.podracer/subscriptions

Edit ~/.podracer/podracer.conf (/etc/podracer.conf has full options):

    # where to save podcasts
    poddir=$HOME/media/podcasts
    
    # where to look for subscriptions
    subscriptions=$HOME/.podracer/subscriptions
    
    # how many times to retry a download
    retries=5

Edit ~/.podracer/subscriptions and add your favorite feeds:

    Car Talk
    http://www.npr.org/rss/podcast.php?id=510208 car_talk
    
    A Way With Words
    http://feeds.waywordradio.org/awwwpodcast a_way_with_words
    
    This American Life
    http://feeds.thisamericanlife.org/talpodcast this_american_life
    
    The News From Lake Wobegon 
    http://americanpublicmedia.publicradio.org/podcasts/xml/prairie_home_companion/news_from_lake_wobegon.xml lake_wobegon
    
    The Writer's Almanac
    http://www.npr.org/rss/podcast.php?id=510101 writers_almanac
    
    Software Engineering Radio
    http://feeds.feedburner.com/se-radio se_radio

Note that any line not beginning with a feed URL is considered a comment and any tokens after a URL are taken to be the download directory for that feed.

If you want avoid downloading all the old episodes from your feeds, run:

    $ podracer -c

This updates podracer's log so that it thinks it has already downloaded all the old podcasts. Finally, edit your crontab and add a line to have podracer fetch new podcasts twice a day:

    # at midnight and noon fetch new podcasts
    0 0,12 * * * /usr/bin/podracer

Now update your crontab with and check it with:

    $ crontab /path/to/your/crontab
    $ crontab -l

All this is based on Matt Cutt's podracer setup post.

Paul Graham and Parody

2009-01-17T12:29:00.000-08:00

I really enjoy reading Paul Graham's essays. He is an excellent writer and often explores simple, interesting hypotheses. For example the idea that succinctness is power in programming languages has a couple of good basic arguments that seem well motivated. Essentially, the idea is that we do not code in assembly because it is a waste of time. We can get basically the same result (or sometimes better) by spending less time writing shorter programs; the principle being that succinctness is the power programming languages provide.

Unfortunately, Paul Graham often extends his approximations past the regime in which they were intended to operate. This has led to an excellent parody, Taste for the Web, and a call of "bullshit" in Dabblers and Blowhards. Both are humorous reads if you're familiar with Graham's essays.

The History of the Microcomputer

2009-01-16T08:53:00.000-08:00

Ever wonder why the stack grows down? How the first single chip computers came about? Why x86 is little endian? Well, then you should definitely check out The History of the Microcomputer -- Invention and Evolution by Stanley Mazor. Not exactly a gripping read, but great historical value.

Vim Single Character Insert

2009-01-14T08:45:00.000-08:00

Here's a nice mapping to add to your .vimrc for single character insert from command mode in vim:

    " single character insert
    nmap <Space> i_<Esc>r

Now to insert the character C to the position under the cursor simply hit the space bar and type C. Any existing characters are shifted to the right. The way it works is pretty neat: it inserts an underscore and then starts a command to replace a single character. Thus when you type the character you want, it replaces the underscore inserted by the mapping. Credit for this goes to Nathan Goulding.

Sed Reference

2009-01-13T08:38:00.000-08:00

This sed page is a great introduction and reference for using sed. In The Unix Programming Environment, Kernighan and Pike relate the tale of Lee McMahon hacking sed together using the ed source. I always thought that story was a great example of the benefits derived from *nix tradition of having access to the source.

Cramming More Components Onto Integrated Circuits

2009-01-12T11:15:00.000-08:00

The original Moore's Law paper by Gordon Moore is a pretty fast, enjoyable read. It's amazing how long the trend he predicted in 1965 has held up. Note that Moore talks about linear circuits which I guess are some kind of analog circuit.

Harbor Freight

2009-01-10T14:08:00.000-08:00

Harbor Freight looks like a great place to get supplies for projects. Examples:

45 Watt Solar Panel
Industrial Suction Cups (think scaling a skyscraper)
Motion Detection Soap Dispenser
Various Electrical Switches

There is also a lot of neat junk for presents.

GNU Make Automatic Variables

2009-01-08T15:35:00.000-08:00

Somehow, the page for make's automatic variables is never very high on my searches. Here it is once and for all: GNU Make Automatic Variables.

Incentivizing IPv6

2009-01-08T13:24:00.000-08:00

To make the transition to IPv6 happen, there needs to be a killer app that demands IPv6. A group of individuals has decided to create that incentive: http://www.ipv6experiment.com/

Hilarious.

Volume Control With amixer and osd_cat

2009-01-07T08:56:00.000-08:00

This script controls volume from the shell, although I typically invoke it with keybindings in xmonad. By default, the PCM channel is selected, but an argument changes which channel the script controls. The use of xosd is due to R.

    #!/usr/bin/env bash
    
    CHANNEL=PCM
    
    if [ "$1" = "-h" ] || [ "$1" = "--help" ] ; then
      p=$(basename $0)
      cat <<HERE
    
      Usage: $p <channel> <option>
      where option is in {+, -, m, u} or is a percent
      e.g. $p 50       # sets $CHANNEL to 50%
           $p Front m  # mutes the channel Front
    
    HERE
      exit 0
    fi
    
    if [ $# -eq 2 ] ; then
      CHANNEL=$1
      shift
    fi
    
    function vol_level {
      amixer get $CHANNEL |\
      grep 'Front Left:'  |\
      cut -d " " -f 7     |\
      sed 's/[^0-9]//g'
    }
    
    function vol_osd {
      killall osd_cat &> /dev/null
      osd_cat -d 2 -l 2 -p bottom -c green \
        -f '-*-terminal-*-*-*-*-*-*-*-*-*-*-*-*' \
        -T "Volume ($CHANNEL)" -b percentage -P $(vol_level) &
    }
    
    function osd {
      killall osd_cat &> /dev/null
      echo $* |\
      osd_cat -d 2 -l 1 -p bottom -c green \
        -f '-*-terminal-*-*-*-*-*-*-*-*-*-*-*-*' &
    }
    
    case "$1" in
      "")
        vol_level
        ;;
      "+")
        amixer -q set $CHANNEL 5%+
        vol_osd
        ;;
      "-")
        amixer -q set $CHANNEL 5%-
        vol_osd
        ;;
      "m")
        amixer -q set $CHANNEL mute
        osd $CHANNEL mute
        ;;
      "u")
        amixer -q set $CHANNEL unmute
        osd $CHANNEL unmute
        ;;
      *)
        amixer -q set $CHANNEL $1%
        ;;
    esac

Hex Editor Via xxd

2009-01-06T09:33:00.000-08:00

xxd is an external program in the vim package that provides hex dumping and hex dump parsing. Because of the parsing, xxd allows you to use any editor as a hex editor, even a stream editor:

    $ xxd foo.pbm | sed 's/ffff/5555/g' | xxd -r > bar.pbm

In general, to use xxd and EDITOR as a hex editor for file FOO, one would:

    $ xxd FOO > BAR
    $ EDITOR BAR
    $ xxd -r BAR > FOO

Note that while you are editing the intermediate file produced by xxd, you can ignore the first and last columns of the file. So in the example above, while you are editing BAR you can ignore the line number and ASCII columns. You can also use xxd to perform hex editing directly in vim by running xxd over the buffer.

Linux CompCert Build

2009-01-05T14:46:00.000-08:00

CompCert only supports Max OS X, but can trivially be built for Linux:

Install Coq and OCaml.
Get the latest source from the CompCert download page.
Unpack the tarball and cd into it.
./configure -prefix ~/projects/compcert

Edit Makefile.config and remove any occurrence of '-arch ppc', i.e.

    PREFIX=/home/ztatlock/projects/compcert
    BINDIR=$(PREFIX)/bin
    LIBDIR=$(PREFIX)/lib/compcert

    CC=gcc
    CPREPRO=gcc -U__GNUC__ -E
    CASM=gcc -c
    CLINKER=gcc
    LIBMATH=
    ARCHOS=x86_LINUX

    #CC=gcc -arch ppc
    #CPREPRO=gcc -arch ppc -U__GNUC__ -E
    #CASM=gcc -arch ppc -c
    #CLINKER=gcc -arch ppc
    #LIBMATH=
    #ARCHOS=x86_LINUX

make all
make install

The resulting ccomp will not generate useful executables (unless you have a cross compiler and emulator handy), but you can get the PPC assembly with ccomp -S.

Amtrak Police Abuse Photographer

2009-01-04T15:43:00.000-08:00

Duane Kerzic was injured and detained by Amtrak police for taking photos from within a station. Other railways are banning photography, which is probably not Constitutional. From browsing comments and a few quick searches, it seems that many people are getting arrested for taking images and video from public locations.

FSF Member

2009-01-01T15:54:00.000-08:00

A tremendous amount of important work is built on top of free software. I tend to ramble on this topic quite a bit, so for the holidays my parents got me a membership to the FSF:

It is about time I joined. Thanks Mom and Dad!

Gordian's Knot

2009-01-01T15:38:00.000-08:00

My brother-in-law got me a ThinkFun Gordian's Knot puzzle for the holidays. The name is taken from a legend involving Alexander the Great, and even inspired some interesting mathematical programming. Here are the rainbow colored (ROYGBIV) pieces:

and the assembled puzzle:

This took some hours to disentangle and reconstruct, very fun and well made! I really believe that anyone could solve the puzzle. It is simply a matter of refusing to quit until you have found the solution (thanks mja!).

The pictures above were taken on Anne and my brand new PowerShot SD790 IS which we got from her parents for the holidays. So far it seems really excellent.

txt2img

2008-12-31T08:10:00.000-08:00

It's nice to generate a png from stdin, e.g. when putting an email address on the web. pbmtext basically provides this, but here's a small wrapper with good default options and the ability to generate any image type that the excellent imagemagick package can handle:

#!/usr/bin/env bash

pbmtext -builtin fixed -space 1 -lspace 5 > $$.pbm
convert $$.pbm $1
rm $$.pbm

Overall, I'm a big fan of the simple formats provided by the netpbm project.

Hat Puzzles

2008-12-30T09:50:00.000-08:00

There seem to be several great hat puzzles. Two good ones (cite):

Puzzle 1: The king blindfolds three ministers, places a white or black hat on each one and removes the blindfolds. "If you can see a black hat, raise your hand", he says. They each raise a hand. "If you can tell what color your hat is, raise your hand." All the hands drop. Then suddenly, one minister raises his hand. What color was his hat?

Puzzle 2: The king has four ministers. He shows them a table with two white hats and two black ones. He blindfolds them, and places a hat on each. Then, he puts one on one side of a wall, and the other three in a line on the other side, facing the wall, like so:
        A  |  B  C  D
A and B can see nothing but the wall. C can see B's hat. D can see B's hat and C's hat. Assuming the ministers are good thinkers, will someone be able to determine their hat color?

Prisoners and Hats Puzzle
Hat Puzzle

cycle and uncycle

2008-12-30T09:04:00.000-08:00

This snippet is for rotating the lines of a file. The script should be invoked as cycle or uncycle and works on either stdin or a single file argument. cycle prints lines 2 through N and then prints the first line. uncycle prints line N and then prints lines 1 through N-1. Obviously, each of these functions is the other's inverse.

#!/usr/bin/env bash

function cycle {
  tail -n +2 $1   # print lines 2 to N
  head -n  1 $1   # print line 1
}

function uncycle {
  tail -n  1 $1   # print line N
  head -n -1 $1   # print lines 1 to N -1
}

f=$(basename $0)

if [ $# -eq 0 ] ; then # output to stdout
  tf=$(tempfile)
  cat > $tf
  $f $tf
  rm $tf
elif [ $# -eq 1 ] && [ -f $1 ] ; then # perform in place
  tf=$(tempfile)
  $f $1 > $tf
  mv $tf $1
else
  echo "ERROR: bad args"
  exit 1
fi

Put this in ~/bin as cycle and then symlink uncycle to it:

  $ wget http://cs.ucsd.edu/~ztatlock/share/cycle -O ~/bin/cycle
  $ ln -s ~/bin/cycle ~/bin/uncycle

Examples:

Popular Science / Mechanics Archives

2008-12-22T14:44:00.000-08:00

Google has archives from over 100 years of Popular Science and Popular Mechanics.

Prank: Stealth USB CapsLocker

2008-12-20T18:12:00.000-08:00

This could be immensely hilarious: usb capslocker.

youtube-dl : Download YouTube Videos

2008-12-19T11:42:00.000-08:00

youtube-dl (2008.11.01) is a nice bit of python to suck down youtube videos so that you watch them, say, on a plane. Found on this list of Linux tools to download video from the web.

Fine Tuning Tex

2008-12-17T15:32:00.000-08:00

Getting a paper into camera ready form can be a massive pain. RJ showed me a great trick to help with the fine tuning:

latex foo.tex
kdvi foo.dvi
Click Settings -> Configure KDVI -> DVI Specials.
Select your favorite editor from the Editor drop down.
Click Apply and then OK.

Now you can middle click anywhere in the DVI and it will open your editor to the correct position in the correct source file. This can save a lot of time when you're removing those dangling, one-word lines in order to meet a page limit.

Holiday Lambda Lights

2008-12-13T18:13:00.000-08:00

My lab's way of decorating for the holidays:

SR Latch to Debounce Switch

2008-12-12T09:23:00.000-08:00

Pushbutton switches are nice for clocking circuits, but the signal is extremely bouncy (ie. it really goes high, low, and in between many times each push). This can be fixed with an SR Latch, which looks like:

and can be implemented with a 74HC00 as:

This past week I went to OSDI 08 with R and some of the UCSD systems guys. It was a lot of fun and very interesting. R and I went to Fry's on the Saturday before the conference and restocked my electronics kit, largely for the LED Display.

ANSI C Quoting in Bash

2008-12-11T08:35:00.000-08:00

Getting the right number of backslashes to properly escape strings in the shell can be a pain. This is particularly true when you want control codes for things like color. A nice way to express such strings is the ANSI C Quoting mechanism in bash. Essentially the raw ANSI C string foo will just be $'foo'.

Vim Command Mode Keybinding

2008-12-10T17:33:00.000-08:00

Use jj to change from insert mode to command mode more quickly in vim. Just add

    imap jj <ESC>

to your vimrc. Now typing jj while you're in insert mode is the same as hitting escape. Courtesy of pmr.

Control Power Outlet

2008-12-05T08:34:00.000-08:00

SparkFun has a very well written tutorial on using a relay to control a power outlet (pdf). This would allow one to control all their appliances with bash. R said that he found some remote control power outlets (presumably IR) that could be hacked as well, but a quick googling turned up nothing.

The LED display project has got me thinking about hardware hacking again. Hack a Day should provide an endless supply of projects if I do decide to rebuild my lab.

Interactive Map

2008-12-04T10:03:00.001-08:00

Sometimes it is useful to transform, rename, or otherwise change a file based on properties that are hard to extract programmatically. A pretty good solution is to loop over the files, display each one, and then read input to determine what to do. For example, if I only want images containing a particular person this would work:

 for p in *.jpg ; do
   display $p
   read -p "remove $p [y/N] ? " ans
   if [ "$ans" == "y" ] ; then
     rm $p
   fi
 done

A more general form:

 #!/bin/bash

 disp=$1
 func=$2
 shift 2
 fs=$*

 for f in $fs ; do
   $disp $f
   read -p "$func $f [y/N] ? " ans
   if [ "$ans" == "y" ] ; then
     $func $f
   fi
 done

Voila! We now have an interactive version of map in bash. I have the above script saved as mapInter on my path, which allows me to do things like:

 $ mapInter display rm $(find . -name '*.png' -or '*.jpg')

to interactively remove files after I have viewed them. Note that regular old map is just:

 #!/bin/bash

 func=$1
 shift 1
 fs=$*

 for f in $fs ; do
   $func $f
 done

Finally, pointers to the shift and read built-ins since I always forget exact details. Higher-order functions in bash FTW!

Frink

2008-12-03T16:47:00.000-08:00

Frink is a fun language designed for quickly performing back-of-the-envelope style calculations. Importantly, it knows many facts about the natural world (e.g. the density of water). For some examples, check out the amusing sample calculations.

The language is named after a Simpson's character who quoted decades ago:

"I predict that within 100 years, computers will be twice as powerful, ten thousand times larger, and so expensive that only the five richest kings of Europe will own them."
-- Professor John Frink

Length of Longest Line in a File

2008-12-01T18:39:00.000-08:00

wc -L gives the longest line in a file. This is very useful if you want pad all lines of a file up to the same width:

  #!/usr/bin/env bash

  tf=$(tempfile)
  
  cat $* > $tf

  L=$(cat $tf | wc -L)

  awk "{ printf(\"%-${L}s\n\", \$0) }" $tf

Note that by using the tempfile, this script works on a list of file arguments or on standard input. Also, the above version is left aligned. If you want a right aligned version, simply change the line:

  awk "{ printf(\"%-${L}s\n\", \$0) }" $tf

to:

  awk "{ printf(\"%${L}s\n\", \$0) }" $tf

(ie. remove the negative sign in the format string).

Logitech QuickCam in Ubuntu 8.10

2008-11-30T10:52:00.000-08:00

Unfortunately Logitech QuickCams do not work out of the box in Ubuntu 8.10. This thread allowed me to get the camera up and running:

problems with logitech quickcam

Basically, hansdegoede is a hero and wrote a user land library to wrap all the video for linux (v4l) calls into a smart version of the code that solves multiple problems. Thanks!

The short version of the solution:

wget http://people.atrpms.net/~hdegoede/libv4l-0.5.0.tar.gz (mirror)
tar xvzf libv4l-0.5.0.tar.gz
cd libv4l-0.5.0
make
sudo make install
export LD_PRELOAD=/usr/local/lib/libv4l/v4l1compat.so
run your favorite webcam app

The export for LD_PRELOAD must be repeated every time you open a shell that is going to run a webcam app. So, you could do this:

  $ echo export LD_PRELOAD=/usr/local/lib/libv4l/v4l1compat.so >> ~/.bashrc

and every bash session would have the LD_PRELOAD environment variable correctly set for the v4l library. Alternatively, you could write wrappers for each webcam app. e.g. wrapping cheese :

  #!/bin/bash

  export LD_PRELOAD=/usr/local/lib/libv4l/v4l1compat.so
  cheese

You can also use camgrab to get images from the webcam using the shell:

  $ sudo apt-get install camgrab
  $ export LD_PRELOAD=/usr/local/lib/libv4l/v4l1compat.so
  $ camgrab -output foo.jpg

Which should be useful for my upcoming super secret project. Thanks again hansdegoede!

Man Section 2 in Ubuntu

2008-11-29T14:22:00.000-08:00

To get section 2 of the man pages in Ubuntu:

  $ sudo apt-get install manpages-dev

I cannot believe section 2 is excluded by default. WTF?!

Extract File Extension

2008-11-29T10:18:00.000-08:00

Here's a useful bash function that invokes awk to get a file's extension:

  # yield the file extension or nothing if there is no extension
  function ext {
    echo $1 | awk 'BEGIN { FS="." } { if(NF > 1) {print $NF} }'
  }

It is useful if you want to convert file type but keep the name. For example:

  for i in $(find . -name '*.png' -or -name '*.jpg' -or -name '*.gif') ; do
    convert "$i" "$(basename $i .$(ext $i)).ppm"
  done

Reverse the lines of a file: rev vs. tac

2008-11-24T21:19:00.000-08:00

"Reverse the lines of a given file." is an ambiguous command. The two obvious parsings are:

Reverse the order lines appear in a file. For example:
```
    0
    1
    2
    3
```
should become:
```
    3
    2
    1
    0
```
This is done by tac. (Get it? "cat" in reverse... haha).
Reverse the order of characters in each line of the file. For example:
```
    0123
    abcd
```
should become:
```
    3210
    dcba
```
This is done by rev.

Bash vs. Python

2008-11-22T11:18:00.000-08:00

Someone whom I respect very much has been making the claim that Python renders BASH obsolete. I frequently make similar assertions, and it's interesting to be on the other side of a such a strong statement. Many of my personal heroes consistently make strong claims that rub many people the wrong way. They still manage to impact history and change the world for the better.

At any rate, here is an example where bash shines and I doubt python can match: create a directory of 150x150 px thumbnails of all the images below the current directory.

    #!/bin/bash
    
    if [ ! -d "thumbs" ] ; then
        mkdir thumbs
    fi

    for i in $(find . -name '*.jpg' -or -name '*.png' -or -name '*.gif') ; do
        convert -resize 150x150 "$i" "thumbs/$(basename $i)"
    done

This is the sort of thing I dash off a dozen times a day. Even if it were possible to express this as easily in Python, I would have to look at several pages of python documentation to get the correct incantations out of os.*. Furthermore, if I want to extend the above script to set permissions, start the search at a given dir, or generate thumbs for only certain directories (eg. by timestamp or name) then simple calls to standard shell tools make the addition trivial. The same extensions in Python involve more documentation grepping and more code. The shell has been tuned for this sort of simple extensibility over a long a period of time.

To be clear, I am a big fan of Python. If the problem you're solving (1) requires real data structures and (2) does not need to be correct, then python is great. If you need real data structures and correctness, go to ML. If you're doing something non-critical that is filesystem based then bash is a good choice.

Awk Primer

2008-11-21T10:41:00.000-08:00

The Awk Primer is a great intro tutorial and reference. I used it to first learn awk along with the also great Awk Tutorial Introduction. Awk's name comes from the authors: Alfred Aho, Peter Weinberger, and Brian Kernighan. But I always forget the damn "W" one. It cost us the CS Trivia competition vs. the faculty last year. It's weird because PJW is actually pretty cool, at one time heading CS research in the legendary Bell Labs. He worked on many of the classic Unix tools and hackers even put his face on a water tower.

One Line Bash Functions

2008-11-19T18:56:00.000-08:00

In a one line bash function, the last command must be followed by a semicolon. So this is OK:

  function hello { echo "Hello World"; }

while this is not:

  function hello { echo "Hello World" }

Reference here.

jUploadr

2008-11-08T10:09:00.000-08:00

jUploadr is a really useful tool for uploading pictures to a Flickr account. A big plus is that it works on all major platforms (all the other uploaders are Windows / OS X only). Unfortunately, you MUST drag and drop photos onto jUploadr, which is a pain when you're running xmonad and have thrown away the mouse.

Make sure to check out the UCSD CSE Flickr photostream.

Thesaurus in Vim

2008-11-02T18:06:00.000-08:00

Ever working on TeX in vim and really wish you could automatically get synonyms for the word under the cursor? Inspired by this post over at the Daily Vim, I added just such a feature.

Step 1: Write the thesaurus script:

  #!/usr/bin/env bash

  firefox http://thesaurus.reference.com/search\?q=$1

Save it somewhere on your PATH, and make it executable (chmod +x thesaurus).

Step 2: Bind K to the thesaurus script. In vim simply issue the command:

  :set keywordprg=/home/ztatlock/bin/thesaurus

Obviously, you should replace 'home/ztatlock/bin' with the path to where your copy of thesaurus lives.

Conway's Game of Life in a Quine

2008-10-30T18:24:00.001-07:00

Parse Command Line Args

2008-10-30T09:38:00.000-07:00

One can write their own argument parsers with while, case, and shift. However, getopt and getopts provide a faster and more robust solution. getopts just "does the right thing" and has saved me considerable effort the past few days. It is really worth RTFM and saving yourself time and headache.

Like many great shell tools, getopts uses its own DSL for input. In this case, you simply provide a string that indicates what flags your script expects (eg. "vrd" to accept "-v", "-r", "-d", or combinations like "-rdv") and insert a colon after a flag that should have an argument (eg. "i:" if you expect "-i arg"). A simple example :

#!/usr/bin/env bash

while getopts "abc:" OPT ; do
  case "$OPT" in
    "a") echo "handling flag -a" ;;
    "b") echo "handling flag -b" ;;
    "c") echo "handling flag -c with arg $OPTARG" ;;
    *)   echo "invalid argument"; exit 1 ;;
  esac
done

Running it:

Notice that getopts will complain if it sees unrecognized arguments.

Color Shell Script

2008-10-25T07:13:00.000-07:00

Following up on the previous post, here is a general script to change the color of certain words in a file or from stdin:

#!/usr/bin/env bash

BKPAT=black
RDPAT=red
GRPAT=green
YLPAT=yellow
BLPAT=blue
MGPAT=magenta
CYPAT=cyan
WTPAT=white

# ANSI color codes
BK=$'\033[1;30m' # black
RD=$'\033[1;31m' # red
GR=$'\033[1;32m' # green
YL=$'\033[1;33m' # yellow
BL=$'\033[1;34m' # blue
MG=$'\033[1;35m' # magenta
CY=$'\033[1;36m' # cyan
WT=$'\033[1;37m' # white
NC=$'\033[0m'    # no color

sed -e "s/$BKPAT/${BK}${BKPAT}${NC}/g" \
    -e "s/$RDPAT/${RD}${RDPAT}${NC}/g" \
    -e "s/$GRPAT/${GR}${GRPAT}${NC}/g" \
    -e "s/$YLPAT/${YL}${YLPAT}${NC}/g" \
    -e "s/$BLPAT/${BL}${BLPAT}${NC}/g" \
    -e "s/$MGPAT/${MG}${MGPAT}${NC}/g" \
    -e "s/$CYPAT/${CY}${CYPAT}${NC}/g" \
    -e "s/$WTPAT/${WT}${WTPAT}${NC}/g" $*

Here is an example of colorize in action:

If you want to use this for a specific purpose, just copy the code and edit the PAT variables to correspond to your desired coloring. For example, if I wanted to make FAIL show up in red, I would simply change the line:

RDPAT=red

to:

RDPAT=FAIL