How To Organize A Clojure Project And Its Dependenciesclojure
People coming from opinionated frameworks like Ruby on Rails often become disoriented in a Clojure project. Rails, being a framework, has the code already organized for you. You just had to learn where to put your models, views, and controllers.
Clojure doesn’t have a major opinionated framework like Ruby on Rails that forces you to follow its conventions.
A lot of beginners get confused and have questions like the following:
- How do you organize a Clojure project?
- What happens when you want to split a module from your application into a separate library?
- How do you work with your own libraries locally?
Let’s explore our options.
Project File Structure
Leiningen by default generates
test directories for
you. Clojure code can go in either directory and it’ll know where to
find and load them.
You can customize this behavior and change it in
One useful thing I like to do is have a
dev source directory only
:dev leiningen profile. This folder would have repl
utilities and maybe saved data for my tests.
Another use of
:source-paths is when you add Clojurescript to your
project and want to separate the code bases. You could make a
test-cljs directory and then add those entries to
Check out the handy dandy leiningen sample project.clj here.
At a namespace level, it’s preferable to have many small namespaces that do one specific thing and avoid having a single namespace that deals with multiple concerns.
Even with that in mind, it’s almost impossible to avoid this
completely. Just about every Clojure project I’ve worked with end up
with an “junk drawer” namespace named
util that had a swiss army
knife’s worth of functions in it.
Breaking Namespaces Up
An issue for people coming from object oriented languages is knowing when is it worth breaking something into a separate namespace. My advice is to just group functions logically by similar overall concerns. It’s probably not the answer you were looking for, but I feel people coming from OOP just need to know it’s okay to group functions arbitrarily.
Namespaces are “cheap” compared to objects so you shouldn’t be afraid to use them. Unlike OOP, you don’t have to manually re-balance an entire inheritance hierarchy tree if you end up needing to move functions around like you would with object methods. In Clojure, just move the function and refer to it in its new location.
Another common question is on directory and file structure. How should you structure a project that does MVC and has models, views, controllers?
This question often comes from people that have worked with Rails and Django where there are best practices and conventions to your directory structure. In this situation, people would ask: Should I make a directory for models, views, and controllers? Or should I make a directory for each model, view, controller and group like that?
Similarly to the previous section on breaking up namespaces I’ll tell you it’s okay to organize your project in whatever way makes logical sense to you. You get to make the decision and choose the tradeoffs that make sense for you, without having a framework do it for you. There’s no established convention that a large group of people are using.
If you want to look for inspiration, look to how the Duct Framework organizes files.
Requires and Imports
Within a namespace, you should always require the functions you need or alias the namespace instead of fully qualifying the namspace/function.
clojure.set/intersection, require the set namespace and
Some of you may be wondering why I say to use
use. Use has been deprecated and discouraged. It provided the exact
same functionality as a
require :refer :all and only confused new
It’s also helpful to alphabetize your requires and imports if possible and if your IDE/editor supports it. It’ll make it easier for future readers to scan through quickly to see if something is already required.
Namespaces should flow well; the things further down are dependent on the things at top and just above it.
Public functions should be at the bottom.
Break your functions into logical sections and groupings within a namespace. For instance, keep all a web application’s handlers in one group separated by a comment block.
For applications with state, I’m a big fan of Stuart Sierra’s component library over mount. It feels more idiomatic to me than mount because it doesn’t rely on global singletons. If you’re starting a new project from the ground up, it’s easy to integrate it into your application.
Component forces you to organize and architect your entire application to use it to its fullest potential.
What you end up doing is splitting your application into components that have internal state, for example, a web server or database component. Then you specify dependencies between them and it’ll figure out how to start, restart, and stop your application in the right order.
Component also helps you make information that you feel should be a global singleton, like a database connection, into something that is just only accessible by the functions that need them.
Dependencies and Libraries
Often you would be developing an application and have a set of common functionality/utilities that would be useful in other projects at your organization.
Of course, the answer is the extract this set of functionality to its own library. The real question is how do you do this and how does it affect your workflow?
Extract The Functionality
Start by extracting the functionality you want and its associated tests into it’s own Clojure project.
You can install your project locally to your
.m2 directory by running
install while in the library’s directory. Then you can require this library
like any other application by going to that application’s
Note: A common mistake I see developers make is constantly using
lein install after making a change and restarting the repl of the
project that depends on the library. There’s a better way with
checkout dependencies. I talk about this below.
Setup a Remote Repository
Your library needs to be accessible somewhere in order for other developers to download it when they clone your project. If you have an open source library, then you can just deploy it to clojars.
However if you need the code to be private, I recommend using the s3-wagon-private plugin to create a remote repository to deploy your artifacts to.
Let’s take a quick step back and talk about the
Clojure puts a
.m2 folder in your home directory that holds your dependencies
(jar and pom files). This directory is where leiningen first looks for
dependencies before downloading them.
The order leiningen looks for dependencies goes like this:
- Local repository (.m2)
- Remote repositories (defaults and based on remotes you’ve configured)
- Optionally, if you have a checkouts directory in your project
folder, leiningen will look for directories with
project.cljfiles and load the code from there.
A lot of people seem learn about leiningen’s behavior for the local and remote repositories, but not about checkout dependencies.
Leiningen Checkout Dependencies
The key change to your workflow when you have a lot of libraries that you’d like
to work on currently without running a
lein install every time you make a
change and restarting your REPL is to use checkout
Essentially, checkout dependencies are a local, checked out version of the dependency from your project.
What you do is at the root level of your primary/application’s directory (the
project.clj in it), you make a directory called
there, you symlink the dependencies you have locally into this folder so that
you end up with a
checkouts/my-utils structure in which the
directory contains its own
Now you can start your REPL and work on your application and jump to the source
of your checkout dependencies and edit, save, and reload the code into your REPL
without ever stopping to
lein install and reinstall your local changes.
Automate Library Deployment and Releases
On the master branch of the library you’re extracting, keep the version number
-SNAPSHOT. This really just indicates that the version of the
library is latest from master.
You shouldn’t really touch the actual version number of the library in
project.clj manually. Whenever you need to do a release and version bump, the
lein release task will do it for you. You can find out more about lein
release on the leiningen
The most ideal scenario is to have your continuous integration software do releases for you at the press of a button.
It turns out that if you don’t make releases of dependent libraries as quick and painless as possible, people are very hesitant to make trivial changes and instead make those changes within the application instead. I’ve seen this myself first hand and am in the process of creating Jenkins jobs for all my libraries to solve this problem.
If the default
lein release task doesn’t suit you or your company’s workflow,
you can customize it per project yourself. Read the docs I linked above to find