Technology

Technology The New Normal How We Work Customer Stories Testing | All Topics

Focus on spec: Combining Specs with `and`

Clojure's new spec library provides the means to specify the structure of data and functions that take and return data. In this series, we'll take one Clojure spec feature at a time and examine it in more detail than you can find in the spec guide.

In our last post, we explored the simplest specs: predicate functions and sets. In this post we'll look at how you can start to combine specs using the and spec.

Welcome to clojurescript.org

We are happy to announce that ClojureScript now has an official web site at http://clojurescript.org! Most of the content from the ClojureScript wiki has been migrated into the new site and organized. 

The site design was carried over from the Clojure web site - thanks to Tom Hickey for the design on the original site. We have adopted the community CLJS logo as the official logo for ClojureScript - many thanks to the designers Chris Oakman and Brett Darnell. 

The new site content is hosted in a GitHub repository and is open for contributions. All contributions require a signed Clojure Contributor Agreement. This repository will accept contributions via pull request and issues with GitHub issues. The contribution and review process is described in more detail on the site contribution page.

This site is a starting point. Because most of the content originated in the wiki, it's likely to need updates in a number of places. There are also many places that content can be added in the Reference, Tools, Guides, and Community sections. We welcome your contributions and thank you for being part of the ClojureScript community! If you have questions, please file an issue on the site repo or contact us on the mailing lists, Slack, IRC, etc for discussion.

We look forward to seeing the site grow!

Clojure spec Screencast: Customizing Generators

One benefit of Clojure specs is that they automatically provide data generators that produce values conforming to the spec which can be used for testing. In addition, you can compose your own generator to more precisely match your data model.

Focus on spec: Predicates

Clojure's new spec library provides the means to specify the structure of data and functions that take and return data. All specs definitions are ultimately based on predicates, which are nothing more than Clojure functions that take a value and return a value that is treated as logically true or false.

Clojure spec Screencast: Testing

Clojure spec defines specifications for both data and functions. In addition to validity checking, specs can generate random samples of the data they specify. This capability enables an alternative to unit testing known as generative, or property-based, testing.

Clojure spec Screencast: Leverage

The new Clojure spec library provides support for data and function specification. In this first in a series of screencasts, Stuart Halloway discusses how spec provides leverage to achieve many returns for a small investment in describing your functions with spec.

State of Clojure 2015 Survey Results

Clojure is a "robust, practical and fast programming language" whose original design goals were aimed at giving developers "succinctness, flexibility and productivity". It turns out that most organizations today are interested in achieving those very goals - as the landscape evolves, they need technologies that help them move faster, respond more effectively, and to take advantage of new architectures and infrastructure. In short, they need simple tools that help them be more agile. 

In this, the sixth consecutive year of the State of Clojure survey, we are seeing the evidence that Clojure is moving across the chasm from a niche tool used by explorers and hobbyists to becoming a critical part of the commercial development landscape, and that its adoption is spreading both broadly among companies, but also more deeply within them.

Before we get to the results, we'd like to first thank everyone who took the time to respond to the survey this year. Response was up more than 75% this year, with 2,445 total responses to analyze. This kind of data is invaluable to the people and organizations using (or considering) Clojure today. As always, we'd also like to thank Chas Emerick for starting the survey back in 2010.

Clojure 1.8

We are pleased to announce the release of Clojure 1.8.

Some of the new features for 1.8 are:

  • More string functions in clojure.string (portable to ClojureScript): index-of, last-index-of, starts-with?, ends-with?, includes?
  • Compiler direct linking - improves performance and startup time
  • Socket server and socket server REPL - adds the ability to allow remote Clojure REPL connections

For more information, see the complete list of all changes since Clojure 1.7 for more details.

Contributors

Thanks to all of those who contributed to Clojure 1.8 (first time contributors in bold):

  • Alexander Yakushev
  • Alex Miller
  • Alex Redington
  • Alf Kristian Stoyle
  • Ambrose Bonnaire-Sergeant
  • Andrew Rosa
  • Andy Fingerhut
  • Andy Sheldon
  • Aspasia Beneti
  • Blake West
  • Bozhidar Batsov
  • Daniel Compton
  • Erik Assum
  • Gary Fredericks
  • Ghadi Shayban
  • Gordon Syme
  • Howard Lewis Ship
  • Jean Niklas L'orange
  • Jeremy Heiler
  • Jonas Enlund
  • Jozef Wagner
  • Karsten Schmidt
  • Kevin Downey
  • Mark Simpson
  • Michael Blume
  • Nahuel Greco
  • Nicola Mometto
  • Nikita Prokopov
  • Nola Stowe
  • Ragnar Dahlén
  • Ralf Schmitt
  • Rich Hickey
  • Russ Olsen
  • Shogo Ohta
  • Steve Miner
  • Stuart Halloway
  • Timothy Baldridge
  • Tsutomu Yano
  • Yanxiang Lou

2015 State of Clojure Community Survey

It's time for the annual State of Clojure Community survey!

If you are a user of Clojure, ClojureScript, or ClojureCLR, we are greatly interested in your responses to the following survey:

State of Clojure 2015

The survey contains four pages:

  1. General questions applicable to any user of Clojure, ClojureScript, or ClojureCLR
  2. Questions specific to the JVM Clojure (skip if not applicable)
  3. Questions specific to ClojureScript (skip if not applicable)
  4. Final comments

The survey will close December 18th. Afterwards we will release all of the data and some analysis.

Many thanks to Chas Emerick for starting and keeping this survey going for so many years!

Extending Transit

The transit spec defines several ground types that are supported by
JSON and MessagePack, on which transit piggy-backs to convey data from
one process to another. These include scalar types like strings and
numbers, and composite types like maps and arrays.

The spec also defines several extension types, which are built from
ground types, each represented by a tag and a value. The tag is a
string, which is a scalar ground type. The value can be a scalar or
composite type. When the reader encounters a composite, e.g. an array,
it unpacks its values and reads each one, recursing until it reaches
scalar ground types.

We use the same extension system to support custom types that are
not part of the transit spec. These can be domain-specific types like
Person or Account, or they can be generic datatypes like trees, sorted
sets, tuples, etc. They don't need to be supported by all of the
languages in our system. In fact, they don't even need to be recognized
by the transit libs running in all of the processes in our system!

Clojure 1.7 is now available

We are pleased to announce the release of Clojure 1.7. The two headline features for 1.7 are transducers and reader conditionals. Also see the complete list of all changes since Clojure 1.6 for more details.

Transducers

Transducers are composable algorithmic transformations. They are independent from the context of their input and output sources and specify only the essence of the transformation in terms of an individual element. Because transducers are decoupled from input or output sources, they can be used in many different processes - collections, streams, channels, observables, etc. Transducers compose directly, without awareness of input or creation of intermediate aggregates.

Many existing sequence functions now have a new arity (one fewer argument than before). This arity will return a transducer that represents the same logic but is independent of lazy sequence processing. Functions included are: map, mapcat, filter, remove, take, take-while, drop, drop-while, take-nth, replace, partition-by, partition-all, keep, keep-indexed, map-indexed, distinct, and interpose. Additionally some new transducer functions have been added: cat, dedupe, and random-sample.

Transducers can be used in several new or existing contexts:

  • into - to collect the results of applying a transducer
  • sequence - to incrementally compute the result of a transducer
  • transduce - to immediately compute the result of a transducer
  • eduction - to delay computation and recompute each time
  • core.async - to apply a transducer while values traverse a channel

Portable Clojure and Reader Conditionals

It is now common to see a library or application targeting multiple Clojure platforms with a single codebase. Clojure 1.7 introduces a new extension (.cljc) for files that can be loaded by Clojure and ClojureScript (and other Clojure platforms). 

There will often be some parts of the code that vary between platforms. The primary mechanism for dealing with platform-specific code is to isolate that code into a minimal set of namespaces and then provide platform-specific versions (.clj/.class or .cljs) of those namespaces.

To support cases where is not feasible to isolate the varying parts of the code, or where the code is mostly portable with only small platform-specific parts, 1.7 provides Reader Conditionals

Reader conditionals are a new reader form that is only allowed in portable cljc files. A reader conditional expression is similar to a cond in that it specifies alternating platform identifiers and expressions. Each platform is checked in turn until a match is found and the expression is read. All expressions not selected are read but skipped. A final :default fallthrough can be provided. If no expressions are matched, the reader conditional will read nothing. The reader conditional splicing form takes a sequential expression and splices the result into the surrounding code.

Contributors

Thanks to all of those who contributed patches to Clojure 1.7:

  • Timothy Baldridge
  • Bozhidar Batsov
  • Brandon Bloom
  • Michael Blume
  • Ambrose Bonnaire-Sergeant
  • Aaron Cohen
  • Pepijn de Vos
  • Andy Fingerhut
  • Gary Fredricks
  • Daniel Solano Gómez
  • Stuart Halloway
  • Immo Heikkinen
  • Andrei Kleschinsky
  • Howard Lewis Ship
  • Alex Miller
  • Steve Miner
  • Nicola Mometto
  • Tomasz Nurkiewicz
  • Ghadi Shayban
  • Paul Stadig
  • Zach Tellman
  • Luke VanderHart
  • Jozef Wagner
  • Devin Walters
  • Jason Wolfe
  • Steven Yi

Also, continued thanks to the total list of contributors from all releases.

Analysis of the State of Clojure and ClojureScript Survey 2014

Yesterday, we posted the raw results from the 2014 State of Clojure and ClojureScript survey, where you can find not only the raw results but the methodology and other details of how it was conducted.  I want to first thank Chas Emerick for having launched the survey back in 2010 and repeating it through 2013, and for reaching out to Alex to run it this year when he could not.  

As always, the purpose of the survey is to shed some light on how and for what Clojure and ClojureScript are being adopted, what is going well and what could stand improvement.   

What's the overview?

We'll look at the individual questions below, but there are some demonstrable trends we can tease out of these responses.  

  1. Clojure (and ClojureScript) are seeing increasing use for commercial purposes, with noticeable growth on all measures where this survey tracks such things.  From use on commercial products and services, to "I use it at work", we're seeing strong positive movement.
  2. ClojureScript is coming along for the ride - even though it does not seem to have a substantial independent identity separate from Clojure, it is also seeing strong growth in commercial application.
  3. The community is adding new users faster each year, which could imply accelerating growth (though remember that this is not a scientific survey).

Let's look at the questions individually, first from the Clojure survey:

How would you characterize your use of Clojure/ClojureScript/ClojureCLR today?

The percentage of respondents using Clojure at work has nearly doubled since the 2012 survey (38% to 65%), which is the big news here.  This would seem to comport with the changes we see in the domains question, and a continued sign of robust commercial adoption of the platform.

In which domains are you applying Clojure/ClojureScript/ClojureCLR?

Web development is still the top dog, and that helps explain the continued increase in usage of ClojureScript as well.  What is significant is the jump in respondents working on commercial products and services (jumping from the low 20s to the low-to-mid 30s), while NoSQL and math/data analysis took a small tumble, essentially reversing positions with commercial development. Network programming is the only other thing to make a substantial move (dropping down about 10%).  Really the takeway is that commercial development is gaining  steadily, demonstrating a continued growth of Clojure in commercial settings, with a quite dramatic increase from 2012 (12% and 14%, respectively).

How long have you been using Clojure?

While the answer distribution has remained largely the same, the relative growth of the "Months" response (moving up a slot) matches up with other metrics, like Kovas Boguta's post analyzing GitHub metrics, to show a continued picture of accelerating growth in the development community.   

Do you use Clojure, ClojureScript, or ClojureCLR?

The JVM platform clearly dominates, which is no shock.  There is no cross-over in the responses between Clojure and ClojureCLR. However, a quite impressive 54.9% of respondents are also using ClojureScript, which is a measurable increase since 2013, though there is still no significant sign in the survey results of a ClojureScript-only userbase.  This seems to imply a continuation of the theme from last time: ClojureScript is adopted (in growing numbers) by existing Clojure developers, not as an independent entity.

We have not previously had ClojureCLR explicitly included in this survey. The recent release of Arcadia (ClojureCLR + Unity gaming engine) may have spurred some recent interest in the ClojureCLR platform. Thanks as always to David Miller's tireless efforts in this area. 

What is your *primary* Clojure/Script/CLR dev environment?

Cursive (a Clojure IDE built on IntelliJ) is the big winner, jumping dramatically to second place.  Interesting that Light Table saw absolute growth in both respondents and percentage, but still fell a spot due to Cursive's massive growth. While Emacs continues to dominate, it is great to see a vibrant collection of options here, to suit every developer's and team's needs.

Which versions of Clojure do you currently use in production or development?

Great to see that 1.6 dominates and it would seem that everyone is able to keep up with the new releases.  A full 18% are using the 1.7 alphas in production or development already.

What version of the JDK do you target?

These answers comport with other survey results recently released, showing a rapid uptake of 1.8 at 49% of respondents.  1.7 is still the most common platform at 70%, and 1.6 is slowly fading at only 14%. Last year, 1.6 was still 19% of the sample, while 1.8 was a mere 5%.  

What tools do you use to compile/package/deploy/release your Clojure projects?
 
Leiningen would now appear to be ubiquitous, at a whopping 98% of respondents using it (up from 75% last year).  There isn't a significant change anywhere else.  

What has been most frustrating for you in your use of Clojure?

There has been remarkably little motion in this list over the years.  Staffing concerns, which jumped all the way to #2 last year, fell a spot this year, falling behind documentation/tutorials.  It is interesting to note that #4, finding editing environments, remains steady even though there has been dramatic shifts and growth within the editor responses.  Otherwise, hard to see any new trends here. Congratulations to everyone for "unpleasant community interactions" continuing to come in dead last.  

Next, let's look at the ClojureScript survey

How would you characterize your use of ClojureScript today?

Once again, we see a dramatic jump in usage at work - from 28% to 49% in just the last year. Serious hobby also climbed roughly 20%.  It would appear that the rising tide floats all boats, as the entire Clojure ecosystem is seeing growth in commercial development use.  

Which JavaScript environments do you target?

Browsers are now ubiquitous, being targeted by 97% of the community, with everything else on the list remaining largely unchanged.

Which ClojureScript REPL do you use most often?

Chas was quite distressed last year to note that more than a fourth of the respondents used no REPL at all.  This year, that number is now almost a third. On the other hand, Austin took a major jump all the way to #2 at 22%, probably due to his commitment to it after last year's survey.  Light Table also came from literally nowhere (not named on last year's survey) to occupy the third spot.  In fact, it wasn't even included in the original responses until a bunch of people requested it be added, so it might be under-represented in the list. So even though even more people aren't using a REPL at all, the options seem to be growing.

What has been most frustrating for you in your use of CLJS?

Through the change in the question style, we can get a better picture of the real answers here.  The difficulty of using a REPL jumps from 14% in 2013 to a whopping 68% this year, while debugging generated JavaScript rises from 14% to 43%.  This is a much better window into how much pain those two items cause the community.  It is impossible to tell because of this change in survey methodology, but the addition of CLJS source map support has most likely made that issue less difficult for many.

The Takeaway

While not a scientific survey, with five years of data it is really possible to spot trends.  Clojure has clearly transitioned from exploratory status to a viable, sustainable platform for development at work.  And as the community continues to add new users at an accelerating pace, we should only expect that trend to continue. 

Our thanks to everyone who took the time to fill out the survey - this kind of sharing is invaluable to the wider community. 

Results of 2014 State of Clojure and ClojureScript Survey

Update 10/24/14: this was the raw results, you can see the analysis of the results here.

The 2014 State of Clojure and ClojureScript Survey was open from Oct. 8-17th. The State of Clojure survey (which was applicable to all users of Clojure, ClojureScript, and ClojureCLR) had 1339 respondents. The more targeted State of ClojureScript survey had 642 respondents.

Reports with charts from some of the survey questions and links to the raw data (see "Export Data" in upper right corner of each report) are here:

Those reports contain charts for all but the grid-style and text response questions.

You can find all of the text responses (extracted and sorted) for the text questions here:

You may wish to refer back to the 2013 or 2012 survey results as well!

2014 State of Clojure & ClojureScript Survey

For the past four years, Chas Emerick has run an annual State of Clojure survey (expanded last year to cover ClojureScript as well). Due to recent happy arrivals in the Emerick household, Chas has asked Cognitect to run the survey this year.

The survey has been broken into two parts. A link to the second survey will appear after you submit the first, or you can use these links directly:

The surveys will be open until October 17th. Shortly thereafter we will release all of the data and some analysis.

If you're not yet planning to attend the Clojure/conj in Washington DC, Nov 20-22, tickets are on sale now (regular registration rate ends Oct. 17th)!

Transducers are Coming

Transducers are a powerful and composable way to build algorithmic transformations that you can reuse in many contexts, and they're coming to Clojure core and core.async.

Transit

We are pleased to announce today the initial release of Transit.

Transit is a format and set of libraries for conveying values between applications written in different programming languages. The key objectives of Transit are to support:

  • Sending values between applications
  • written in different programming languages
  • without requiring schemas/context
    • i.e., to be self-describing at the bottom
  • with extensibility
  • and good performance
  • with reach to the browser

JSON currently dominates similar use cases, but it has a limited set of types, no extensibility, and is verbose. Actual applications of JSON are rife with ad hoc and context-dependent workarounds for these limitations, yielding coupled and fragile programs.

On the other hand, the reach of JSON is undeniable. High performance parsers are widely available. Thus Transit is specified as an encoding to and from both JSON and MessagePack, a binary JSON-like format with widely available parsers. In particular, both formats have parsers written in C for languages like Ruby and Python that reach to C for performance.

Transit supports a minimal but rich set of core types:

  • strings
  • booleans
  • integers (to 64 bits w/o truncation)
  • floats
  • nil/null
  • arrays
  • maps (with arbitrary scalar keys, not just strings)

Transit also includes a wider set of extension types:

  • timestamps
  • UUIDs
  • URIs
  • arbitrary precision integers and decimals
  • symbols, keywords, characters
  • bytes
  • sets
  • lists
  • hypermedia links
  • maps with composite keys

Transit is extensible - users can define extension types in exactly the same way as the included extension types.

The emphasis of Transit is on communication between programs, thus it prioritizes programmatic types and data structures over human readability and document orientation. That said, it does have a readable verbose mode for JSON.

Transit is self describing using tags, and encourages transmitting information using maps, which have named keys/fields which will repeat often in data. These overheads, typical of self-describing formats, are mitigated in Transit by an integrated cache code system, which replaces repetitive data with small codes. This yields not only a reduction in size, but also an increase in performance and memory utilization. Contrast this with gzipping, which, while it may reduce size on the wire, takes time and doesn't reduce the amount of text to be parsed or the number of objects generated in memory after parsing.

We are shipping an 0.8 version of the Transit spec, which has extensive documentation for implementors, as well as interoperable implementations for:

We welcome feedback and suggestions on the transit-format list.

I'd like to thank the team at Cognitect that built Transit:

  • Tim Ewald - Team Lead
  • Brenton Ashworth
  • Timothy Baldridge
  • Bobby Calderwood
  • David Chelimsky
  • Paul deGrandis
  • Benoit Fleury
  • Michael Fogus
  • Yoko Harada
  • Ben Kamphaus
  • Alex Miller
  • David Nolen
  • Russ Olsen

I hope you find Transit useful, and look forward to your feedback.

Rich