You are previewing Clojure Programming.

Clojure Programming

Cover of Clojure Programming by Chas Emerick... Published by O'Reilly Media, Inc.
  1. Clojure Programming
  2. SPECIAL OFFER: Upgrade this ebook with O’Reilly
  3. Preface
    1. Who Is This Book For?
      1. Engaged Java Developers
      2. Ruby, Python, and Other Developers
    2. How to Read This Book
      1. Start with Practical Applications of Clojure
      2. Start from the Ground Up with Clojure’s Foundational Concepts
    3. Who’s “We”?
      1. Chas Emerick
      2. Brian Carper
      3. Christophe Grand
    4. Acknowledgments
      1. And Last, but Certainly Far from Least
    5. Conventions Used in This Book
    6. Using Code Examples
    7. Safari® Books Online
    8. How to Contact Us
  4. 1. Down the Rabbit Hole
    1. Why Clojure?
    2. Obtaining Clojure
    3. The Clojure REPL
    4. No, Parentheses Actually Won’t Make You Go Blind
    5. Expressions, Operators, Syntax, and Precedence
    6. Homoiconicity
    7. The Reader
      1. Scalar Literals
      2. Comments
      3. Whitespace and Commas
      4. Collection Literals
      5. Miscellaneous Reader Sugar
    8. Namespaces
    9. Symbol Evaluation
    10. Special Forms
      1. Suppressing Evaluation: quote
      2. Code Blocks: do
      3. Defining Vars: def
      4. Local Bindings: let
      5. Destructuring (let, Part 2)
      6. Creating Functions: fn
      7. Conditionals: if
      8. Looping: loop and recur
      9. Referring to Vars: var
      10. Java Interop: . and new
      11. Exception Handling: try and throw
      12. Specialized Mutation: set!
      13. Primitive Locking: monitor-enter and monitor-exit
    11. Putting It All Together
      1. eval
    12. This Is Just the Beginning
  5. I. Functional Programming and Concurrency
    1. 2. Functional Programming
      1. What Does Functional Programming Mean?
      2. On the Importance of Values
      3. First-Class and Higher-Order Functions
      4. Composition of Function(ality)
      5. Pure Functions
      6. Functional Programming in the Real World
    2. 3. Collections and Data Structures
      1. Abstractions over Implementations
      2. Concise Collection Access
      3. Data Structure Types
      4. Immutability and Persistence
      5. Metadata
      6. Putting Clojure’s Collections to Work
      7. In Summary
    3. 4. Concurrency and Parallelism
      1. Shifting Computation Through Time and Space
      2. Parallelism on the Cheap
      3. State and Identity
      4. Clojure Reference Types
      5. Classifying Concurrent Operations
      6. Atoms
      7. Notifications and Constraints
      8. Refs
      9. Vars
      10. Agents
      11. Using Java’s Concurrency Primitives
      12. Final Thoughts
  6. II. Building Abstractions
    1. 5. Macros
      1. What Is a Macro?
      2. Writing Your First Macro
      3. Debugging Macros
      4. Syntax
      5. When to Use Macros
      6. Hygiene
      7. Common Macro Idioms and Patterns
      8. The Implicit Arguments: &env and &form
      9. In Detail: -> and ->>
      10. Final Thoughts
    2. 6. Datatypes and Protocols
      1. Protocols
      2. Extending to Existing Types
      3. Defining Your Own Types
      4. Implementing Protocols
      5. Protocol Introspection
      6. Protocol Dispatch Edge Cases
      7. Participating in Clojure’s Collection Abstractions
      8. Final Thoughts
    3. 7. Multimethods
      1. Multimethods Basics
      2. Toward Hierarchies
      3. Hierarchies
      4. Making It Really Multiple!
      5. A Few More Things
      6. Final Thoughts
  7. III. Tools, Platform, and Projects
    1. 8. Organizing and Building Clojure Projects
      1. Project Geography
      2. Build
      3. Final Thoughts
    2. 9. Java and JVM Interoperability
      1. The JVM Is Clojure’s Foundation
      2. Using Java Classes, Methods, and Fields
      3. Handy Interop Utilities
      4. Exceptions and Error Handling
      5. Type Hinting for Performance
      6. Arrays
      7. Defining Classes and Implementing Interfaces
      8. Using Clojure from Java
      9. Collaborating Partners
    3. 10. REPL-Oriented Programming
      1. Interactive Development
      2. Tooling
      3. Debugging, Monitoring, and Patching Production in the REPL
      4. Limitations to Redefining Constructs
      5. In Summary
  8. IV. Practicums
    1. 11. Numerics and Mathematics
      1. Clojure Numerics
      2. Clojure Mathematics
      3. Equality and Equivalence
      4. Optimizing Numeric Performance
      5. Visualizing the Mandelbrot Set in Clojure
    2. 12. Design Patterns
      1. Dependency Injection
      2. Strategy Pattern
      3. Chain of Responsibility
      4. Aspect-Oriented Programming
      5. Final Thoughts
    3. 13. Testing
      1. Immutable Values and Pure Functions
      2. clojure.test
      3. Growing an HTML DSL
      4. Relying upon Assertions
    4. 14. Using Relational Databases
      1. clojure.java.jdbc
      2. Korma
      3. Hibernate
      4. Final Thoughts
    5. 15. Using Nonrelational Databases
      1. Getting Set Up with CouchDB and Clutch
      2. Basic CRUD Operations
      3. Views
      4. _changes: Abusing CouchDB as a Message Queue
      5. À la Carte Message Queues
      6. Final Thoughts
    6. 16. Clojure and the Web
      1. The “Clojure Stack”
      2. The Foundation: Ring
      3. Routing Requests with Compojure
      4. Templating
      5. Final Thoughts
    7. 17. Deploying Clojure Web Applications
      1. Java and Clojure Web Architecture
      2. Running Web Apps Locally
      3. Web Application Deployment
      4. Going Beyond Simple Web Application Deployment
  9. V. Miscellanea
    1. 18. Choosing Clojure Type Definition Forms Wisely
    2. 19. Introducing Clojure into Your Workplace
      1. Just the Facts…
      2. Emphasize Productivity
      3. Emphasize Community
      4. Be Prudent
    3. 20. What’s Next?
      1. (dissoc Clojure 'JVM)
      2. 4Clojure
      3. Overtone
      4. core.logic
      5. Pallet
      6. Avout
      7. Clojure on Heroku
  10. Index
  11. About the Authors
  12. Colophon
  13. SPECIAL OFFER: Upgrade this ebook with O’Reilly
O'Reilly logo

Vars

You’ve already used and worked with vars a great deal. Vars differ from Clojure’s other reference types in that their state changes are not managed in time; rather, they provide a namespace-global identity that can optionally be rebound to have a different value on a per-thread basis. We’ll explain this at length starting in Dynamic Scope, but first let’s understand some of their fundamentals, since vars are used throughout Clojure, whether concurrency is a concern or not.

Evaluating a symbol in Clojure normally results in looking for a var with that name in the current namespace and dereferencing that var to obtain its value. But we can also obtain references to vars directly and manually dereference them:

map
;= #<core$map clojure.core$map@501d5ebc>
#'map                               1
;= #'clojure.core/map
@#'map
;= #<core$map clojure.core$map@501d5ebc>
1

Recall from Referring to Vars: var that #'map is just reader sugar for (var map).

Defining Vars

Vars make up one of the fundamental building blocks of Clojure. As we mentioned in Defining Vars: def, top level functions and values are all stored in vars, which are defined within the current namespace using the def special form or one of its derivatives.

Beyond simply installing a var into the namespace with the given name, def copies the metadata[146] found on the symbol provided to name the new (or to-be-updated) var to the var itself. Particular metadata found on this symbol can modify the behavior and semantics of vars, which we’ll enumerate here.

Private vars

Private vars are a basic way to delineate parts of a library or API that are implementation-dependent or otherwise not intended to be accessed by external users. A private var:

  1. Can only be referred to using its fully qualified name when in another namespace.

  2. Its value can only be accessed by manually deferencing the var.

A var is made private if the symbol that names it has a :private slot in its metadata map with a true value. This is a private var, holding some useful constant value our code might need:

(def ^:private everything 42)

Recall from Metadata that this notation is equivalent to:

(def ^{:private true} everything 42)

We can see that everything is available outside of its originating namespace only with some effort:

(def ^:private everything 42)
;= #'user/everything
(ns other-namespace)
;= nil
(refer 'user)
;= nil
everything
;= #<CompilerException java.lang.RuntimeException:
;=   Unable to resolve symbol: everything in this context, compiling:(NO_SOURCE_PATH:0)>
@#'user/everything
;= 42

You can declare a private function by using the defn- form, which is entirely identical to the familiar defn form, except that it adds in the ^:private metadata for you.

Docstrings

Clojure allows you to add documentation to top-level vars via docstrings, which are usually string literals that immediately follow the symbol that names the var:

(def a
  "A sample value."
  5)
;= #'user/a
(defn b
  "A simple calculation using `a`."
  [c]
  (+ a c))
;= #'user/b
(doc a)
; -------------------------
; user/a
;   A sample value.
(doc b)
; -------------------------
; user/b
; ([c])
;   A simple calculation using `a`.

As you can see, docstrings are just more metadata on the var in question; def is doing a little bit of work behind the scenes to pick up the optional docstring and add it to the var’s metadata as necessary:

(meta #'a)
;= {:ns #<Namespace user>, :name a, :doc "A sample value.",
;=  :line 1, :file "NO_SOURCE_PATH"}

This means that, if you want, you can add documentation to a var by specifying the :doc metadata explicitly, either when the var is defined, or even afterward by altering the var’s metadata:

(def ^{:doc "A sample value."} a 5)
;= #'user/a
(doc a)
; -------------------------
; user/a
;   A sample value.
(alter-meta! #'a assoc :doc "A dummy value.")
;= {:ns #<Namespace user>, :name a, :doc "A dummy value.",
;=  :line 1, :file "NO_SOURCE_PATH"}
(doc a)
; -------------------------
; user/a
;   A dummy value.

This is a rare requirement, but can be very handy when writing var-defining macros.

Constants

It is common to need to define constant values, and using top level def forms to do so is typical. You can add ^:const metadata to a var’s name symbol in order to declare it as a constant to the compiler:

(def ^:const everything 42)

While a nice piece of documentation on its own, ^:const does have a functional impact: any references to a constant var aren’t resolved at runtime (as per usual); rather, the value held by the var is retained permanently by the code referring to the var when it is compiled. This provides a slight performance improvement for such references in hot sections of code, but more important, ensures that your constant actually remains constant, even if someone stomps on a var’s value.

This certainly isn’t what we’d like to have happen:

(def max-value 255)
;= #'user/max-value
(defn valid-value?
  [v]
  (<= v max-value))
;= #'user/valid-value?
(valid-value? 218)
;= true
(valid-value? 299)
;= false
(def max-value 500)    1
;= #'user/max-value
(valid-value? 299)
;= true
1

max-value is redefined, after which point valid-value? implements different semantics due to its reliance on our “constant.”

We can prevent such mishaps using ^:const:

(def ^:const max-value 255)
;= #'user/max-value
(defn valid-value?
  [v]
  (<= v max-value))
;= #'user/valid-value?
(def max-value 500)
;= #'user/max-value
(valid-value? 299)
;= false

Because max-value is declared ^:const, its value is captured by the valid-value? function at compile-time. Any later modifications to max-value will have no effect upon the semantics of valid-value? until it is itself redefined.

Dynamic Scope

For the most part, Clojure is lexically scoped: that is, names have values as defined by the forms that circumscribe their usage and the namespace within which they are evaluated. To demonstrate:

(let [a 1
      b 2]
  (println (+ a b))      1
  (let [b 3
        + -]
    (println (+ a b))))  2
;= 3
;= -2
1

a and b are names of locals established by let; + and println are names of vars containing functions defined in the clojure.core namespace, which are available within our current namespace.

2

The local b has been bound with a different value, as has +; since these definitions are more lexically local than the outer local binding of b and the original var named +, they shadow those original values when evaluated within this context.

The exception to this rule is dynamic scope, a feature provided by vars. Vars have a root binding; this is the value bound to a var when it is defined using def or some derivative, and the one to which references to that var will evaluate in general. However, if you define a var to be dynamic (using ^:dynamic metadata),[147] then the root binding can be overridden and shadowed on a per-thread basis using the binding form.

(def ^:dynamic *max-value* 255)
;= #'user/*max-value*
(defn valid-value?
  [v]
  (<= v *max-value*))
;= #'user/valid-value?
(binding [*max-value* 500]
  (valid-value? 299))
;= true

Note

Dynamic vars intended to be rebound with binding should be surrounded with asterisks — like *this*—also known as “earmuffs.” This is merely a naming convention, but is helpful to alert a reader of some code that dynamic scope is possible.

Here we are able to change the value of *max-value* outside of the lexical scope of its usage within valid-value? by using binding. This is only a thread-local change though; we can see that *max-value* retains its original value in other threads:[148]

(binding [*max-value* 500]
  (println (valid-value? 299))
  (doto (Thread. #(println "in other thread:" (valid-value? 299)))
    .start
    .join))
;= true
;= in other thread: false

Dynamic scope is used widely by libraries and in Clojure itself[149] to provide or alter the default configuration of an API without explicitly threading context through each function call. You can see very practical examples in both Chapters 15 and 14, where dynamic scope is used to provide database configuration information to a library.

Visualizing dynamic scopeTo illustrate, consider a var: it has a root value, and for each thread, it may have any number of thread-local bindings, which stack up as nested dynamic scopes come into effect via binding.

A var holding a single root value, and many thread-local stacks of thread-local bindings

Figure 4-5. A var holding a single root value, and many thread-local stacks of thread-local bindings

Only the heads of these stacks may be accessed (shown bolded above). Once a binding is established, the prior binding is shadowed for the duration of the dynamic scope put into place by binding. So within the innermost dynamic scope here, *var* (and therefore, (get-*var*)) will never evaluate to :root, :a, or :b:

(def ^:dynamic *var* :root)
;= #'user/*var*
(defn get-*var* [] *var*)
;= #'user/get-*var*
(binding [*var* :a]
  (binding [*var* :b]
    (binding [*var* :c]
      (get-*var*))))
;= :c

Each level of dynamic scope pushes a new “frame” onto the stack for the var being bound:

The effect of establishing a new thread-local binding for a var using binding

Figure 4-6. The effect of establishing a new thread-local binding for a var using binding

(binding [*var* :a]
  (binding [*var* :b]
    (binding [*var* :c]
      (binding [*var* :d]
        (get-*var*)))))
;= :d

We’ve seen how dynamic scope can be used to control the behavior of functions at a distance, essentially allowing callers to provide an implicit argument to functions potentially many levels down in a call tree. The final piece of the puzzle is that dynamic scope can also work in reverse, to allow functions to provide multiple side-channel return values to callers potentially many levels up in a call tree.

For example, while Clojure provides some incredibly convenient IO functions to simply retrieve the content of a URL (e.g., slurp and others in the clojure.java.io namespace), such methods provide no easy way to retrieve the corresponding HTTP response code when you require it (a necessary thing sometimes, especially when using various HTTP APIs). One option would be to always return the response code in addition to the URL’s content in a vector of [response-code url-content]:

(defn http-get
  [url-string]
  (let [conn (-> url-string java.net.URL. .openConnection)
        response-code (.getResponseCode conn)]
    (if (== 404 response-code)
      [response-code]
      [response-code (-> conn .getInputStream slurp)])))

(http-get "http://google.com/bad-url")
;= [404]
(http-get "http://google.com/")
;= [200 "<!doctype html><html><head>..."]

That’s not horrible, but as users of http-get, this approach forces us to deal with the response code for every call in every context, even if we aren’t interested in it.

As an alternative, we could use dynamic scope to establish a binding that http-get can set only when we’re interested in the HTTP response code:

(def ^:dynamic *response-code* nil)                                    1

(defn http-get
  [url-string]
  (let [conn (-> url-string java.net.URL. .openConnection)
        response-code (.getResponseCode conn)]
    (when (thread-bound? #'*response-code*)                            2
      (set! *response-code* response-code))                            3
    (when (not= 404 response-code) (-> conn .getInputStream slurp))))  4

(http-get "http://google.com")
;= "<!doctype html><html><head>..."
*response-code*
;= nil
(binding [*response-code* nil]
  (let [content (http-get "http://google.com/bad-url")]
    (println "Response code was:" *response-code*)
    ; ... do something with `content` if it is not nil ...
    ))
;= Response code was: 404
;= nil
1

We define a new var, *response-code*; users of http-get opt into accessing the response code it obtains by binding this var.

2

We use thread-bound? to check if the caller of http-get has established a thread-local binding on *response-code*. If not, we do nothing with it.

3

set! is used to change the value of the current thread-local binding on *response-code* so that the caller can access that value as desired.

4

Now that http-get can use the optional dynamic scope around *response-code* to communicate auxiliary information to its callers, it can simply return the string content loaded from the URL instead of the compound vector of [response-code url-content] (assuming the URL is not 404).

Again, to illustrate:

image with no caption

Because set! acts on a var’s binding by replacing the current thread-local value, a caller within the dynamic scope established by binding—whether a direct one or one 50 frames up the call stack—can access that new value without it having been threaded back through the return values of all the intervening function calls. This works for any number of vars, any number of bindings, and any number or type of set!-ed values, including functions. Such flexibility enables simple API extensions like auxiliary returns as we’ve demonstrated here, up to more elaborate and powerful things like non-local return mechanisms.

Dynamic scope propagates through Clojure-native concurrency formsThe thread-local nature of dynamic scope is useful—it allows a particular execution context to remain isolated from others—but without mitigation, it would cause undue difficulty when using Clojure facilities that by necessity move computation from one thread to another. Thankfully, Clojure does propagate dynamic var bindings across threads—called binding conveyance—when using agents (via send and send-off), futures, as well as pmap and its variants:

(binding [*max-value* 500]
  (println (valid-value? 299))
  @(future (valid-value? 299)))
; true
;= true

Even though valid-value? is invoked on a separate thread than the one that originally set up the dynamic scope via binding, future propagates that scope across to the other thread for the duration of its operation.

Note that, while pmap does support binding conveyance, the same does not hold true for lazy seqs in general:

(binding [*max-value* 500]
  (map valid-value? [299]))
;= (false)

The workaround here is to ensure that you push the dynamic scope required for each step in the lazy seq into the code that will actually be evaluated when values in the seq are to be realized:

(map #(binding [*max-value* 500]
        (valid-value? %))
     [299])
;= (true)

Vars Are Not Variables

Vars should not be confused with variables in other languages. Coming from a language like Ruby, where code usually looks like this:

def foo
  x = 123
  y = 456
  x = x + y
end

It’s incredibly tempting for new Clojure users to try to write code like this:

(defn never-do-this []
  (def x 123)
  (def y 456)
  (def x (+ x y)
  x))

This is very poor form in Clojure. But, what’s the worst that could happen?

(def x 80)
;= #'user/x
(defn never-do-this []
  (def x 123)
  (def y 456)
  (def x (+ x y))
  x)
;= #'user/never-do-this
(never-do-this)
;= 579
x                         1
;= 579
1

“Waitaminute, I declared x to be 80 at the start!”

def always defines top level vars—it is not an assignment operation affecting some local scope. x and y in this example are globally accessible throughout your namespace, and will clobber any other x and y vars already in your namespace.

With the exception of dynamic scope, vars are fundamentally intended to hold constant values from the time they are defined until the termination of your application, REPL, etc. Use one of Clojure’s other reference types for identities that provide useful and proper semantics for changing state in place, if that is what you are looking for. Define a var to hold one of those, and use the appropriate function (swap!, alter, send, send-off, et al.) to modify the state of those identities.

Changing a var’s Root BindingDespite our various warnings against using vars as variables as understood in other languages, there is value in mutating their root bindings occasionally and with great care. To change a var’s root binding as a function of its current value, there’s alter-var-root:

(def x 0)
;= #'user/x
(alter-var-root #'x inc)
;= 1

When the var in question contains a function, this provides a superset of the functionality found in most aspect-oriented programming frameworks. Concrete examples in that vein are provided in Aspect-Oriented Programming and Building mixed-source projects.

You can also temporarily change the root binding of a bunch of vars with with-redefs, which will restore the vars’ root bindings upon exiting its scope; this can be very useful in testing, for mocking out functions or values that depend upon environment-specific context. See Mocking for an example.

Forward Declarations

You can opt not to provide a value for a var; in this case, the var is considered “unbound,” and dereferencing it will return a placeholder object:

(def j)
;= #'user/j
j
;= #<Unbound Unbound: #'user/j>

This is useful for when you need to refer to a var that you haven’t defined a value for yet. This can happen when implementing certain types of algorithms that benefit from alternating recursion—or, you may simply want to have the implementation of a function to come after where it is used as a matter of style or in an attempt to call attention to primary or public API points. Clojure compiles and evaluates forms in the order presented in your source files, so any vars you refer to must at least be declared prior to those references. Assuming such vars’ values are only required at runtime (e.g., if they are placeholders for functions), then you can redefine those vars later with their actual values. This called a forward declaration.

In such cases, the declare macro is somewhat more idiomatic. Using it instead of def alone makes explicit your intention to define an unbound var (rather than leaving open the possibility that you simply forgot to provide a value), and it allows you to define many unbound vars in a single expression:

(declare complex-helper-fn other-helper-fn)                    1

(defn public-api-function
  [arg1 arg2]
  ...
  (other-helper-fn arg1 arg2 (complex-helper-fn arg1 arg2))    2

(defn- complex-helper-fn                                       3
  [arg1 arg2]
  ...)

(defn- other-helper-fn
  [arg1 arg2 arg3]
  ...)
1

We declare our helper functions’ vars before they are referred to.

2

Now we can put our primary/public API near the top of our source file and refer to our helper functions freely.

3

We properly define our helper functions later on in the source file.



[146] See Metadata for a primer on metadata in Clojure.

[147] Attempting to use binding on a var that is not :dynamic will result in an exception being thrown.

[148] Please excuse the momentary slew of Java interop; it is necessary to use a native thread in order to demonstrate this characteristics of dynamic vars. See Using Java’s Concurrency Primitives and Chapter 9 for explanations of what’s going on here.

[149] Examples include *warn-on-reflection* as detailed in Type Hinting for Performance and Type errors and warnings. *out*, *in*, and *err*, and indirect usages of binding, like with-precision are discussed in Scale and Rounding Modes for Arbitrary-Precision Decimals Ops.

The best content for your career. Discover unlimited learning on demand for around $1/day.