Clojure's n00b attraction problem

This weekend I've been diving head-first into the world of Clojure, and before I become too accustomed to it I'd like to put my thoughts down, as they're currently coming from an exceedingly important perspective: that of the total n00b.

I really like Clojure-the-language, just as I love Scheme and Lisp in general, however, you'll note that none of these languages are especially popular, at least when compared to PHP, Python, Ruby, and even Objective-C. I think part of the reason for this is lack of good, "officially sanctioned experiences."

Popularity plays an important role in a language's relevance. It serves as an indicator to potential adopters as to whether they'll find a strong community behind it that can answer all of their questions, and whether it will have all the frameworks and libraries they might need.

Over the years, I've observed three factors that influence how likely a language is to become popular:

Evangelists—a language that's just starting out needs to have great evangelists, the more the better. These are people who write blog posts, create the first libraries and frameworks, and give speeches about how awesome their chosen language is. Clojure has this, and then some.
Relevance—does the language meet some unsatisfied need? Clojure certainly does. One of its primary reasons for existence is to solve the parallel-computing problem in an efficient and simple manner. It also satisfies the need for a modern, powerful, expressive language.
An insanely great, officially sanctioned user experience, and this is where I think Clojure fails miserably.

Languages grow by convincing n00bs to spend time and effort learning the ropes, and thereby turning these n00bs into code-warrior-evangelists who attract other n00bs, and so on. This makes their perspective exceedingly important.

I've been following Clojure's progress for over a year now as an outsider, and I know from observation that the community at large is aware that they have a n00b attraction problem. It would be hard not to notice the sheer volume of getting-started related questions on the Clojure Google Group.

Having spent the last 24 hours diving into Clojure-land, I can offer the following thoughts:

Clojure needs to improve its officially-sanctioned experience

Most popular languages today, whether it's Python, Ruby (On Rails, or not), PHP, Java, C# or even Objective-C, make it very easy to get started because they have very specific recommendations to make on choice of development environments (IDEs), documentation, and the hoops you must jump through to use other people's code in your projects.

Specifically, they have:

Great/Clear, officially sanctioned documentation. I've found Clojure's to be terrible in comparison. Good documentation groups similar functionality together in a clear way. It has copious example code alongside the API docs, and there are links to related API. (See the documentation for PHP, Java, and newLISP for great examples of this).
Great/Simple, officially sanctioned IDEs and/or source editors. Clojure has good IDE support (after trying all of them, I've currently settled on IntelliJ), but the getting-started experience is pretty terrible (more on this in the next section).
Simple, officially sanctioned methods/systems for using third-party libraries. Ruby has their gem system, Objective-C is completely different with its frameworks and Xcode, but they both satisfy the requirement of telling n00bs exactly what to do.

The "officially sanctioned" part is important because it tells newcomers exactly where to turn, and it minimizes the amount of effort that goes into competing endeavors, thereby focusing that effort onto the officially sanctioned one.

Once a n00b has a working Clojure setup, they'll be able to branch out and explore other options, but throwing hundreds of poor choices at them will only overwhelm them and scare them away.

Clojure suffers from choice paralysis, and bad choices

The Clojure community recently made great progress in becoming more n00b-friendly with the creation of an officially-sanctioned Getting Started guide. This is an excellent resource, but it suffers from choice paralysis.

There's something wrong when you have an officially-sanctioned Getting Started guide, and yet your n00bs still end up spending hours getting started.

When a similar point was brought up on the Clojure mailing list, some defended Clojure's current state of affairs, saying that it's super simple to get up and running with Clojure, that it takes mere minutes:

Here's how I installed the Clojure REPL on my system.
1) Downloaded clojure-1.1.0.zip
2) Unzipped clojure.jar
3) Ran java -jar clojure.jar

The first step is admitting there's a problem, and by that token the above is genuine dishonesty. There are several problems with this:

That is not how you develop with Clojure. Clojure, being Java based, uses packages and namespaces, specially structured directory layouts and numerous build systems. You don't spend all your time playing with a REPL.
Clojure 1.1 is already outdated. The datatype stuff that's in the 1.2 builds is being touted as The Way Things Should Be Done From Now On™. Clojure is a new language, and so far it's an early-adopter's game, and early adopters usually like being on the cutting edge, and not forced to learn and use methodologies that have been essentially deprecated. Many n00bs are aware of this (I was), and so they want to use 1.2, and that has serious consequences on the Getting Started Experience. Yes, staying on the bleeding edge, and in this case using different versions of Clojure should be clearly outlined somewhere in the Getting Started Experience.

So, returning to the larger point, how can it take hours or even days to get started with Clojure, despite the Getting Started Guide?

Let's see what a n00b has to go through.

Paralysis: Getting clojure.jar & clojure-contrib.jar

First, you have to obtain Clojure. This isn't that easy. There are a bunch of ways of doing it (if you can't tell, I like lists, could be the Lisper in me):

Get it through the leiningen script, which installs Maven and puts it in Maven's repository. Don't know what WTF Maven is? Scared once you've gotten the gist of it? Where the hell is clojure.jar? Is it safe to reference it once you've found it in "Maven's repository"? Should you make a copy of it? You're not alone, fellow n00b.
Download a pre-built copy from Google Code. Where's Clojure 1.2? Do these jars have the source included in them so that IDEs can do their magic?
Download a pre-built copy from GitHub. Same questions as above.
Learn git if you haven't and clone the repository from GitHub and build it yourself.
Grab the nightly builds. This is actually an excellent resource, and I forgot how I found it.Why isn't it clearly linked to from clojure.org or the Getting Started Guide?

Now, once you've finally managed to obtain your clojure.jar of choice, you're faced with another problem: how do I get ___ IDE/source-editor to use it? How do you use it across projects? This isn't addressed anywhere! For example, the instructions for using Emacs and swank-clojure/slime do not explain this, nor do the instructions for using any of the IDEs, yet it's probably a pretty common question that's asked by n00bs.

Clojure has many Getting Started related issues, and one of them is the fact that unlike other languages, Clojure is a library. With Python, you have a system-wide Python install directory. Heck, with Java, you have a system-wide JDK directory, that everything else uses. Not only that, but you have several of them on OS X, and it's fairly straight-forward to tell and IDE to use one over the other. You also have the concept of a current JDK, which is achieved through a symlink.

With clojure, this is all very strange for people who have no idea how to use Emacs, IntelliJ or any of the other IDEs, and don't know anything about Java class paths and all that.

Paralysis: How do I run this bloody thing?

OK. Now you've spent 30 minutes to an hour (or more) figuring out how to obtain the right copies of clojure.jar and clojure-contrib.jar. You know where they are and you want everything to use them. Now, how do you go about using them?

Most languages have an easily accessible REPL/interpreter:

$ python
Python 2.6.1 (r261:67515, Feb 11 2010, 15:47:53) [GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

$ irb

$ newlisp

newLISP v.10.2.9 on OSX IPv4/6 UTF-8, execute 'newlisp -h' for more info.

To run a ruby/python/newlisp file you just put the name of the file after the executable. Even Java's method, though much more complicated than the above, is still manageable because there is a single, documented, officially sanctioned way of building and running Java files (the commands: javac, jar, java).

How do you do that with Clojure? Surprisingly, the answer is nowhere to be found in the Assembla Getting Started Guide. You have to find the REPL and main page on Clojure.org, and the answer is enough to scare away most n00bs coming to clojure from any of the above languages:

$ java -cp clojure.jar clojure.main /path/to/myscript.clj

Sorry, that doesn't make a nice first-impression on n00bs. So after asking on #clojure or the Google group, someone from the community (read: not an officially sanctioned answer), will suggest that they use a "clj" script. What is that and where do you get it?

A clj script is a script you either obtain from some random person, or you write yourself, and it essentially wraps the above command in some logic so that you have the same experience that you get from irb, python, lua, newlisp, etc. It handles separating arguments passed to the java command from those passed to the script, and it sets up the classpath string (and if you happen to be a n00b to Java, it means you're at an instant disadvantage when it comes to Clojure, because "classpath string" means nothing to you, and it means you've got to learn yet another thing before you're able to get started).

What should be basic functionality that comes with a language is instead left up to the n00b to figure out. I eventually decided to write my own in newlisp.

Paralysis: Emacs/VIM

The fact that Clojure is based on Java is both a blessing and curse. A blessing because you get all those Java libraries for free, and a curse because you get to deal with Java and hell that comes with it (ant, maven, etc.).

After a n00b has obtained Clojure and graduated beyond 'clj' and the REPL, they'll want to edit source code and build jar files of their own. This is probably the point where most n00bs run away in terror, because they will be assaulted by a myriad of choices. This critical point is something that experienced Clojure developers have the least sympathy for, because they are already comfortable with the choices they've made, and they've long since forgotten or stopped caring about the existence of other possibilities (probably because of how overwhelming they are).

The n00b has no such luxury though, they must decide, for example, whether to go with Emacs (or VIM) or an IDE. Today, lots of seasoned Clojure developers will make the unfortunate mistake of recommending Emacs. I say this as a fairly experienced Emacs user (I've written elisp, I've used Aquamacs, terminal emacs, and Cocoa Emacs. I've tricked out my init.el, etc. I still think it's horrible advise to give to n00bs. Same goes for VIM).

As this post is already long enough, I'm not even going to touch the thousands of issues that a n00b will definitely run into if they choose to follow this advise, but needless to say, it's just another way Clojure scares potential adopters.

However, the n00b will likely give Emacs and swank-clojure a try, and they will likely either run away screaming or stick it through and survive the initiation, albeit after days of pain. And they've still yet to write a single line of Clojure!

Paralysis: IDEs

Now, say our n00b is either wise and decides to shun the time-suck that is emacs, or is told to use an IDE. They now have to figure out which IDE to use. There are three options currently advised on the Getting Started page:

NetBeans + Enclojure
Eclipse + Counterclockwise
IntelliJ IDEA + La Clojure

One major problem is that there are no instructions for how to use the clojure.jar/clojure-contrib.jar files you've downloaded/compiled. It would be great if there was!

So far, I'm most impressed with the experience I had with IntelliJ + La Clojure. I found it to be less bloated and faster to launch than the others, and it's very powerful and has lots of useful features (like support for TextMate-like snippets), and plugins. The Clojure plugin is good, but could definitely use work. In general, I liked the overall design of IntelliJ over the others though.

I think the Clojure community should "officially sanction" one of these IDEs and put their combined efforts into making it insanely great. The entire Apple developer community happily uses a single IDE: Xcode. Why? Because there's little reason for an alternative. It's pretty freaking awesome. This makes for a much better n00b experience because there's only one thing to investigate and learn about.

That's not to say the other IDEs shouldn't be supported or listed, but just that from a n00bs perspective, it would be great if there was a single, awesome go-to IDE to dive right in.

Build Systems

Yet another set of choices n00bs are faced with is figuring out how to actually compile their source into an executable. The Clojure community has made some good progress here thanks to the work of Phil Hagelberg (technomancy) on leiningen.

In Java, there are many build systems, and n00bs much figure out whether they'll go with Ant, Gradle, Maven, leiningen, and possibly others.

Oddly enough, the Getting Started docs assume the n00bs trying to get started have heard of these things. They mention them, but they don't actually explain that these are build systems.

Leiningen is pretty nifty once you've figured it out, but unfortunately its tutorial doesn't provide a basic example of how to go from point A (source code) to point B (executable jar file). It does describe the process, but it's not very clear to a n00b who doesn't know all of the project.clj syntax for specifying the :main class and using :gen-class. Nor does it mention how to include a jar file that's located locally on the drive and not obtained through the online repos. These basic things, for some reason, are glossed over, or left out altogether.

It would be great if this entire problem were solved by the IDE transparently via drag & drop (the way Xcode does it), or if there were a simple way to integrate leiningen with them.

Documentation

As mentioned, even if our n00b survives the initiation, Clojure's core documentation is pretty horrid. This is one area that needs much improvement (the importance of examples can't be understated), and I think the community would be wise to seek out other languages like PHP and newLISP for inspiration on how to get this right.

Conclusion

Clojure is a beautiful language, but if its community wants more people to appreciate the language, then it may want to consider making the getting started experience beautiful too. All of the recent changes I've seen are a step in the right direction, and I hope this list of grievances can help to further improve the experience.