Generating Source Files with Leiningen

Recently, we needed to include some generated source files in a project. The source code generation was project specific, so we didn't want to have to create a leiningen plugin specifically for it. To get this to work required using quite a few of leiningen's features.

This post will explain how to use lein to customise you build to generates a source file, but many of the steps are useful to implement any form of lein build customisation.

The Generator

The source code generator is going to live in the my.src-generator namespace. Here's an example, that just generates a namespace declaration for the my.gen namespace under target/generated/my/gen.clj.

(ns my.src-generator
  (:require [clojure.java.io :refer [file]]))

(defn generate []
  (doto (file "target" "generated" "my" "gen.clj")
    (-> #(.getParentFile) #(.mkdirs))
    (spit "(ns my.gen)")))

Development only code

The source generation code should not be packaged in the jar, so we place it in dev-src/my/src_generator.clj, and add dev-src and the generated source directories to the :dev profile's :source-paths. The :dev profile is automatically used by leiningen unless it is producing a jar file. When producing the jar, the dev profile will not be used, so dev-src will not be on the :source-path (we add the generated directory to the base :source-path below).

:profiles {:dev {:source-paths ["src" "dev-src" "target/generated"]}}

Running project specific code with leininingen

The run task can be used to invoke code in your project. To use lein's run task we need to add a -main function to the my.src-generator namespace.

(defn -main [& args]
  (generate))

In the project.clj file we also tell lein about the main namespace. In order to avoid AOT compilation of the main namespace, we mark it with :skip-aot metadata.

:main ^:skip-aot my.src-generator

Customising the jar contents

The generated files need to end up in the jar (and possibly be compiled), so we put them on the :source-paths in the project. If we had wanted to include the sources without further processing, we could have added the generated directory to :resource-paths instead.

:source-paths ["src" "target/generated"]

Extending the build process

Now we can tell lein to generate the source files whenever we use the project. We do this by adding the run task to the :prep-tasks key. Leiningen runs all the tasks in :prep-tasks before any task invoked by the lein command line.

The tricky bit here is that the run task will itself invoke the :prep-tasks, so we want to make sure we don't end up calling the task recursively and generating a stack overflow. To solve this, add a gen profile, and disable the prep tasks in it. We use the :replace metadata to ensure this definition takes precedence. See the leiningen profile documentation for more information on :replace and it's sibling :displace.

:gen {:prep-tasks ^:replace []}

Then use this profile when setting the :prep-tasks key in the project.

:prep-tasks [["with-profile" "+gen,+dev" "run"]  "compile"]

Now when we run any command, the sources are generated.

Adding an alias

Finally we may want to just invoke the source generation, so let's create an alias to make lein gen run the generator. We need the gen profile for this, or otherwise the generator will run twice.

:aliases {"gen" ["with-profile" "+gen,+dev" "run"]}

The final project.clj

For reference, the final project.clj looks like this:

(defproject my-proj "0.1.0-SNAPSHOT"
  :dependencies [[org.clojure/clojure "1.4.0"]]
  :source-paths ["src" "target/generated"]
  :main ^:skip-aot my.src-generator
  :prep-tasks [["with-profile" "+gen,+dev" "run"]  "compile"]
  :profiles {:dev {:source-paths ["src" "dev-src" "target/generated"]}
             :gen {:prep-tasks ^:replace []}}
  :aliases {"gen" ["with-profile" "+gen,+dev" "run"]})

Conclusion

This required using many of lein's features to get working - hopefully you'll find a use for some of them.

Discuss this post here.

Published: 2013-10-28

Pelure