Sunday, August 31, 2014

Switching to Fish Shell

I started using Fish as my primary shell a few months ago. While I like Bash, the promise of a more modern shell intrigued me. I spend entirely too much time on the command line. My affinity for Bash has less to do with its features as a language or shell than with the UNIX philosophy of many small programs which play nicely together.

Fish jokingly bills itself as "a command line shell for the 90s". It isn't revolutionizing what a shell does, rather it starts from a strong design document to provide a better experience. If you're unclear on the difference between a shell, terminal emulator, Bash, & command line interface, try Bryan J. Brown's description on his blog.

What's Good with Fish

Why would I switch to Fish? Immediately after trying it out, a few advantages were apparent. I didn't even have to consult help documentation.

Discovery is where Fish shines. I discovered new, useful programs on Mac OS simply by tabing through available completions. Fish's completion is incredibly smart & detailed; it knows files, commands, variables, & flags. Bash does this too, but Fish is far superior & comes with a huge collection of completions for common programs. It's main advantage is that it'll show options, so the completion is exploratory, whereas in other shells the completion is a just convenience for people who already know what they're looking for. Fish shows the definition of a particular flag, function, or program—as well as the current value of variables—instead of merely showing that they exist.

Many of the tools I use have dozens of flags. I love them, but I can't memorize each flag for every one. Take Ack for example. I usually just add a flag for the programming language I'm searching (e.g. --js) & the string I'm looking for. But the other day I wanted to see the number of matches in each of the large list of files I was searching. Now, I know ack can do this, but I don't know what flag(s) I need. Typically, I'd need to open up ack's man page, search through it, close it, & then run the command. With Fish, I typed a couple dashes, then tab to see all its completions, spotted --count right away, & ran the command without leaving my current context.

Another nice advantage of Fish's completion; it learns from previously typed commands. So even if there's no custom-built completions for a particular program, Fish learns how you use it & develops completions over time.

Fish also has colors! Nice ones! They pop more than I'm used to. What's more, the shell provides convenient abstractions for changing colors. The set_color command lets you use natural language like "red" rather than the crazy looking echo \033[1;33m (yes, this is actually how you change colors in Bash). set_color is handy, but Fish also has added features like prompt_pwd, which is great for shortening the working directory for inclusion in a prompt.

If you don't want to spend hours configuring a custom prompt, Fish comes with a couple dozen nice ones built-in. You can run **fish_config** to open a configuration interface in a web browser which gives you copy-pastable prompt code. This config feature makes it super quick to get started without a ton of research & looking up replacement tokens. Every shell should have such a feature.

Scripting in Fish is far more straightforward, as the shell's language is minimal & clean. It looks Ruby-esque & favors natural language everywhere over strange, punctuated incantations. Because it's a smaller & more rationale language, learning the basics of Fish scripting is quicker than with other shells.

Fish also has wonderful error messages, perhaps the best of any programming language I've dealt with. That may not seem valuable but it helps immensely with learning the shell, especially when transitioning from Bash. Fish will not only point to the erroneous character, but will note common mistakes & try to guess what you missed. For instance, in Bash a subshell is launched with $(…) whereas Fish uses (); the $ in Fish means one & only one thing, that a variable is being used. So when you use a $ in the wrong context, it says so. An example:

> echo $(whoami)
fish: Did you mean (COMMAND)? In fish, the '$' character is only used for accessing variables. To learn more about command substitution in fish, type 'help expand-command-substitution'.
echo $(whoami)

Fish is half written in its own scripting language, so it's easy to see how some features work & extend them. I noticed that there weren't any completions for Node & NPM, so I added them myself by aping existing ones. Exposing so much of the shell's core functionality makes it customizable & approachable.


In a way, Fish is the perfect shell for someone just getting starting at the command line because of its brilliant completions, easy (no code!) configurability, & sane scripting language. Unfortunately, for me, it's not quite perfect because I'm already used to Bash's quirky parts & rely on numerous packages, settings, & scripts that assume a more common (read: Bash) environment.

Example: z. Z is a vital utility for me; it allows me to quickly jump between my current location & places I've been previously. Z's API is simple; "z [string]" where "string" somewhere matches the place you want to go. So if I'm destroying system settings in "/Library/Application Support" & then need to go to my Doge Decimal project, I type "z doge" & am transported to "/Users/phette23/code/dogedc". But Z is a shell script; it's written in Bash. Luckily I found a port for Fish, but for a while I was trying really hacky solutions (including proxying Z through Bash every time I ran it). Other tools, like nvm, pose this same problem.

To be fair; various incompatibilities aren't Fish's fault. They can only be solved by popularity, so when someone writes a script they think "I need this to work in all the popular shells: Bash, Zsh, & Fish". Sublime Text proved to be the biggest compatibility pain. Sublime uses os.environ['PATH'] to find the user's path & this path is used in all kinds of plug-ins. I use several linting plugins, such as SublimeLinter-JSHint, which rely on JSHint being in your path. But Fish separates path locations with a space & not a colon; Sublime consequently misreads the whole PATH string, breaking almost every plugin I've installed.

I found a way around…and it was to default back to Bash. I ran chsh -s /usr/local/bin/bash to switch my default shell back to Bash, so when Sublime runs os.environ['PATH'] it comes back with a predictable, colon-separated path. But then, because I actually want to use Fish, I had to edit all my terminal emulator profiles (I use iTerm2) such that, instead of running as login shells that would default to Bash, they execute the /usr/local/bin/fish command. A surmountable problem, but it took me weeks to identify what was wrong & how to fix it.

In general, Fish users will run into more compatibility problems with all sorts of tools that assume a Bash or strict POSIX environment. As I said, much of this isn't Fish's fault, but it is worth noting that the shell doesn't strive for 100% POSIX compliance. In a way, this is necessary; Fish conflicts with POSIX only where a substantial benefit in usability is at stake. That's great, but it also causes headaches that can't be easily fixed since backwards compatibility is broken.

While Fish breaks with some POSIX traditions, in other places it doesn't go far enough. It relies heavily on double-underscored internal functions; anywhere there's a naming convention like this, there are scoping problems. It's not clear to me why all shell scripting languages lack true objects; everything ends up in the global scope. While Fish has nice arrays, certainly better than Bash, it still lacks data structures that aid in organization. A hash/dict/associative array type is badly needed. I think this might be a place where Windows PowerShell improves upon POSIX shells, though I haven't used PS enough to truly know.

There are also things I genuinely like about Bash. I like its || & && logical operators, which behave slightly different from the natural language or & and of Fish. I like some of Bash's crazy-looking expansions, like !! (references the last command), which are weird & hard to remember but handy at times.

My main struggles with Fish revolve around output redirection, which it seems to be more stringent about. I still haven't found a nice way to quietly test if a command exists (which occurs all throughout my dotfiles, since I try not to assume a particular software setup). In Bash, this was simple with command -v $PROGRAM. But command is a shell built-in, not an external program, & so it differs in Fish. Fish doesn't replicate the "v" flag, it only uses "command" as a way to bypass aliases. I've worked around it with a two-line solution: PROGRAM --version >/dev/null; if test $status…. This runs the program, silencing all output, & then checks the exit status (which would be 0, signifying an error, if the command didn't exist). It works, but it's slower & more verbose.

There's more than you ever wanted to know about my transition to Fish shell. I'm guessing that switching shells isn't something people consider very often. Those who use the command line rarely probably don't think it's worth the trouble (or don't even know/care that it's possible), while those who rely on the command line necessarily build up lots of dependence on a specific environment. Despite all that, I'd strongly recommend Fish to anyone and I thoroughly enjoy using it every day. The pains are, oddly enough, lesser for inexperienced shell users, while the benefits are greater thanks largely to how sane and helpful Fish is designed to be.

Thursday, August 7, 2014

How Not To Do User Testing

  • Perform tests only after a final product has already been rolled out
  • Use your tests to reify assumptions already built into the product
  • Test once and then never again because hey, you’re finished
  • Refuse to accept the validity of any given test until a statistically representative sample of your user populace has been obtained (it’ll never happen)
  • Never change your testing tasks and procedures, even the ones that prove to be deeply flawed, poorly worded, uninformative
  • Ask users for their opinions rather than observing what they actually do. “Do you like this background gradient?” is a particularly apt question.
  • Conversely, test only tasks you think are important without gauging what users think is important
  • Collect personal information and video recordings during tests with no plans for how to secure the data or when to delete it
  • Simply refuse to do user testing

Monday, April 21, 2014

Looping Over Regular Expressions in JavaScript

Much as JavaScript has literal forms for strings & numbers, it also has a literal Regular Expression (henceforth regex) form. So you can wrap characters in single or double quotes to make a string literal, & you can wrap characters in forward slashes ("/") to make a regex literal.

So regexes are literals in JavaScript…except JavaScript is sort of a broken language with regard to literals. The typeof operator is nearly useless.
typeof /foo/
// returns "object"…damn you, typeof
/foo/ instanceof RegExp
// true! hurrah for instanceof
Because regexes have a literal form of sorts, you can do nice things with them like put them in an array & loop over the array:
var tests = [/Foo/i, /BAR/, /baz/i],
    str = 'This is a sentence. Foo, says the sentence.';

// check if each regex has a match in sentence
tests.forEach(function(re) {
    if (re.test(str)) {
        console.log(re + ' is a match!');
This is the approach I use in my Wordpress Spam Clicker bookmarklet: I have an array of regexes matching known spammer patterns which I loop over, testing each comment against them.

BUT what if you want to slightly modify each regex in an array? For instance, what if you want to loop over regexes but test for strings with a space at the beginning the match? You can save some typing & potentially (depending on how big the array of regexes is) a lot of bytes by storing a truncated version of the regexes & then modifying them later. Except it doesn't work:
var tests = [/Foo/i, /BAR/, /baz/i],
    str = 'This is a sentence. Foo, says the sentence.';

// check if each regex has a match in sentence
tests.forEach(function(re) {
    if ((/\s/ + re).test(str)) {
        console.log(re + ' is a match!');
I'm trying to take each regex & prepend the special character for a space ("\s"), so /foo/i should become /\sfoo/i. But the addition operator doesn't work here, JavaScript doesn't know how to add 2 regular expressions, instead it casts them to a string (typeof (/\s/ + /foo/i) === 'string'). What do?

Well, JavaScript also has constructor functions for all its literals: String(), Number(), & RegExp(). Generally, you do not use these. I repeat, if you're writing code like var count = new Number(0) you can stop it, stop it right now. One reason is that typeof count will return "object" if count was created with a constructor. But also it's just an unnecessary amount of typing.

BUT it turns out that compiling regexes from strings can be done using the RegExp constructor. So to achieve my earlier goal I can write:
var tests = ['Foo', 'BAR', 'baz'],
    str = 'This is a sentence. Foo, says the sentence.';

// check if each regex has a match in sentence
tests.forEach(function(re) {
    if (RegExp('\\s' + re).test(str)) {
        console.log(re + ' is a match!');
Instead of storing regexes which are later cast to strings, I can store strings & then essentially cast them to regexes using the RegExp constructor. I have to escape the backslash to ensure \s makes it into the regex, but it works. The RegExp constructor takes the regex flags as a second argument too, so I could write RegExp('\\s' + re, 'i') to make all my regexes case insensitive. This, too, could be very handy & save a lot of bytes/typing.

Sunday, March 30, 2014

Git Tools

On a recent commute I mulled over the various tools I use to make git, the popular distributed version control software, easier to use and more powerful. Here's a round up of what I use, or find interesting.

gitconfig - anything you do with the git config command can also be placed in a .gitconfig runtime configuration file in your home directory. My main use case is aliases that save me a ton of typing. Simple shortcuts like "c = commit -m" are an obvious starting point. But git also has several commands with multiple, handy flags; for instance, my "sweet-looking but concise logs" alias is "l = log --pretty=oneline -n 20 --graph --abbrev-commit". I do not want to memorize and type that monster, not once, not ever. My entire .gitconfig can be found in my dotfiles repo.

gitsh - an interactive shell for git. If you're running a bunch of git commands in a row, enter gitsh & run them without typing "git" over & over. This can be very useful as git commands tend to come in waves; "oh I need to commit these final changes, rebase, checkout master, & then merge this feature branch" is a common workflow, for example. Gitsh also displays repository information—the current branch & working directory status—in its prompt.

hub - a command-line tool for interacting with GitHub. I don't use this because it wouldn't gain me a whole lot of efficiency but I bet hub would be invaluable if you're an active GitHub user.

js-git - an interesting project to implement git in client-side JavaScript with support for various browser storage APIs. Could feasibly bring git into environments like Chromebooks, where one doesn't have command-line access but could still benefit from version control.

Sublime Text Packages

I use Sublime Text as my main editor & these two packages are great in terms of git integration.

Git - this package is essential if you're working in Sublime Text. It gives access to all the common git commands—add, commit, diff, log—right in the command palette. You never have to leave your editor to access version control, you can stay in a single context & do everything you need. It's a huge boon to productivity. I use "Quick Commit" (adds and commits the file I'm currently viewing) all the time. I bet roughly a third of my commits are through that single convenience method.

GitGutter - highlights lines that have been changed, added, or deleted in the file you're viewing with coloring in the gutter. You can select from a few different styles of coloring. This is a small nicety most of the time but can be of great assistance when returning to a project that has a dirty working directory or stashed changes which you've forgotten.

Thursday, March 20, 2014

Start-Up Thinking Is Inappropriate for Libraries

tl;dr — if you believe your institution is a social necessity, start-up thinking is a terrible approach.

A recent conversation with a friend who has worked in the start-up space brought up Brian Mathew's "Think Like a Start-Up" white paper and some unresolved issues I have with it, never publicly articulated. See also: The Marketing Unproblem of Libraries.


Most start-ups fail. Start-ups are praised for their agility, their ability to solve problems, but not for their longevity. If you believe in the worth of libraries as institutions, I'm guessing you don't want 75% of them to go under. It's unfathomably, eye-rollingly ironic that Mathews starts his white paper with doomsaying about the sustainability of academic libraries and then offers transient organizations as a model for survival. I can't even.

Trying to flip this fact later in the white paper does little to assuage my concerns. Noting the failure-prone nature of start-ups is not simply some snarky observation; it speaks to irreconcilable differences between how start-ups are run & how are libraries should be run. If you want your library to be around next year, next decade, next century, you probably don't want to emphasize risk-taking. Long-term thinking might be more suitable. You probably don't want to be a technological solutionist. Heck, you probably don't want to rely on the assumption that you only need to serve a population with access to certain technologies. Making an iPhone app is not enough. Making any app is not enough. Being a community-driven organization just might be enough.

It's also worth mentioning start-up culture has its own atrocities. It's hostile to women.* It's hostile to people of color. They're just generally not the type of organization socially conscious people probably want to work for, not that there aren't exceptions to this generality. I find it intolerable to valorize start-up culture while its downsides go unmentioned.

On Choosing Appropriate Proxies

I envision a rejoinder that libraries should praise & emulate the agility & innovativeness of start-ups, focusing on those attributes rather than their ephemerality. Leaving aside the fact that this straw-person argument is basically "but if you only look at the good things start-ups are good," it hints that start-ups are a poor proxy for what we actually want to talk about. I despise poor proxies. They muddle the debate & obscure the underlying issues. To use my favorite example: when we use age as a proxy for technical savvy, we not only discriminate against older folks but overestimate the abilities of the young. So let's discuss "libraries should be agile & innovative," not "libraries should think like start-ups."

But that's a lame tag-line right? And tag-lines are important. It's catchy, "Think like a Start-Up." But if it's so misleading as to be positively counterproductive, it should be ditched.


Finally, there's perhaps a tension in that start-ups are capitalist institutions par excellence & modern libraries** typically follow a more socialist, resource-sharing approach. But that's too much to go into here & I haven't thought about it enough.

In general, there are virtually no similarities between what libraries should be(come) & what start-ups are. Mic drop.


* There are numerous examples or articles I could have linked to here but Ashe Dryden's is particularly apt. If you think this statement is contestable, leave a comment & I can cite additional instances of hostility.

** Obviously "social libraries" like Benjamin Franklin's Library Company of Philadelphia (unnecessary emphasis mine), where only subscribed members could access the collection, aren't following a very socialist model. These are less common in America today than tax-funded public libraries, for instance.