Transcript 107: Ivan Willig

THE COGNICAST TRANSCRIPTS

EPISODE 107

In this episode we talk to Ivan Willig about using Clojure and Datomic at Clubhouse.io. 

The complete transcript of this episode is below.

The audio of this episode of The Cognicast is available here.

Transcript

CRAIG:    Hello, and welcome to Episode 107 of The Cognicast, a podcast by Cognitect, Inc. about software and the people who create it.  I'm your host, Craig Andera. 

All right, well, I would like to mention to you Operability.io.  Now this is a conference held in London September 19th and 20th, 2016.  It's going to be about dev ops.  This is brought to our attention by our friend Marco Abis, who is one of the people who was initially responsible for EuroClojure, so you know he does good work.  

This is a conference that he's helping to run, and he gave us a heads up.  Took a look at it.  I'm like, yeah, it looks like a thing that our listeners might want to be aware of.  You can check that out at Operability.io if you're going to be in or around or can get yourself to London in mid September.  Check that out.  

I want to mention there's a ClojureBridge happening in Pittsburgh September 9th and 10th.  As you're well aware, if you've listened to the show a bunch of times, ClojureBridge is a great organization.  Its aim is to increase diversity within the Clojure community by offering free, beginner friendly, Clojure programming workshops for women.  Check that out at ClojureBridge.org.  You can see the list of their events there and take a look at the one that's happening in Pittsburgh September 9th and 10th.

Finally, I want to mention Clojure NYC.  This is a user group in New York City.  They are having a meet-up on Wednesday, August 24th, and Nathan Marz will be speaking.  Nathan is a good guy.  He's been a guest on the show.  He's going to be talking about an optimization technique called Inline Caching.  It looks pretty cool based on the description.  You can read more about that at www.meetup.com/clojure-nyc or search for Clojure NYC.  It looks like cool stuff.  I wish I was going to be able to attend that one.  

That's all I've got for you today, though, so we will go ahead and go on to Episode 107 of The Cognicast.  

[Music: "Thumbs Up (for Rock N' Roll)" by Kill the Noise and Feed Me]

CRAIG:    Perfect.  Well then let's begin.  Yeah?

IVAN:    Yeah.

CRAIG:    All right.  All right, everybody.  Welcome.  Today is Thursday, June 23rd, 2016, and this is The Cognicast.  

I am very pleased today to welcome as our guest a developer at Clubhouse.io, Ivan Willig.  Welcome to the show, Ivan.

IVAN:    Thanks for having me on.  

CRAIG:    I should ask.  I hope I pronounced your name correctly.  I forgot to ask before the show if that's right.

IVAN:    You did.  Yeah, you got it - Ivan.

CRAIG:    Okay, great.  Well, we are excited to have you on the show today, Ivan.  You are a Clojurist.  You have interesting things to say about it and about your company's use of that technology, as well as a couple others that people here have heard us talk about before.  

But before we do that, we always ask the guest the question to open the show.  That question is: Would you please relate to us some experience of art for whatever that means to you.  People have talked about concerts they've been to.  They talk about all sorts of things.  We're going to just throw it wide open just for you to share some artistic experience that you've had and share it with our audience, so what have you got for us?

IVAN:    An artistic experience I've had.  I guess when I was in school, I kind of had a hard time.  I was one of those people that didn't do so well in school, I guess.  I think art was always, for me -- I actually did art as an outlet because, I don't know.  I guess I'm not a person who follows rules very well.  Well, sometimes.  I do follow rules, but I'm not very good at them, so I felt art was -- always felt, in school, art was a space where you kind of do what you want, it was a little bit of freedom, and it was creative.  That to me was kind of like an outlet for many years.  

I don't do it any more, which is kind of weird, but definitely that was probably the biggest influence it's had on my life as kind of the act of doing it and a way of getting away because there's not really any right and wrong answers in art.  You can kind of just do stuff and try it out.  It often will not work, and you'll have to do it over again, but I found that kind of interesting.

CRAIG:    Did you have a particular medium that you liked to work in?

IVAN:    I moved around a little bit.  I used to do kind of like painting, watercolors, oils.  I also did colored pencils for a little while, and also my dad was really big into photography, and I got into photography a little bit in high school.  It's kind of strange, and I don't really know why, but I do none of that now and I don't know why.  Maybe it's because I have a job, a life, and stuff.  

CRAIG:    Well, that was actually one of the things I was thinking was when you were saying it was an outlet.  I'm personally really interested in the idea of programming as a creative enterprise, much like, say, painting or working with colored pencils, as you said.  First of all, do you see it that way as well?  And do you think, perhaps, that the reason you don't do it any more is because programming is your creative outlet?  Does that make any sense to you?

IVAN:    Yeah, totally.  I think that's true.  I think, yeah, there's some truth there.  Yeah, it's definitely.  I remember when I was learning how to program, one of my professors taught me or told me, like, people think that programming is not creative, people who don't program, I'd say.  I find it one of the most -- it's similar in art in some ways where you basically get to start with not very much and it's a lot of creativity.  You get to create again without a lot of restrictions.  There are more restrictions than art, obviously, but I think that's true.

Although, I will say that it can be more frustrating than art because you've got to think about what you're doing.  You've got to set up.  You've got to make sure what you're doing is actually correct by the compiler.  You've got to get the database set up, depending on what kind of programming you're doing.  Yeah, I think that's maybe true.  

Also, I have a family now, so I just have less free time in general, so maybe that's it.

CRAIG:    Sure.  Yeah, I can sympathize on that one.  

IVAN:    Yep.

CRAIG:    I think that's a really cool introduction to you personally, but let's maybe shift over to the professional side of it.  I mentioned that you are a developer at Clubhouse.io.  I wonder if you wouldn't mind, by way of introducing us to you and your company.  

IVAN:    Sure.  Yeah, I'm Ivan, and I'm one of the backend engineers at Clubhouse.  Clubhouse is a project management software tool that we are trying to kind of achieve a balance between sophistication and ease of use.  I believe Clubhouse kind of came out of a frustration.  Well, I think we've all experienced, at the company, a frustration with current project management tools, but it definitely came out of a frustration that was experienced by our two founders, Andrew and Kurt.  

There was an ease of use.  Some of the tools out there are very easy to use, but they don't scale very well for large companies or even small companies necessarily.  And then a lot of the other tools are very sophisticated.  You can configure them.  They have a lot of knobs and levers, but they become unwieldy in a different way where you have so many options that it just doesn't scale in large teams very well.  Clubhouse is our attempt to try to strike a balance between that.

CRAIG:    Yeah, so I was looking at your website, Clubhouse.io, and I have not used the product myself, but one of the things I noticed is that clearly somebody is paying attention.  At the very minimum floor of usability is: Does it look like something you would want to use?  I know that's not the same as usability, but it looks like a product where somebody is like, yes, we're definitely paying attention to that kind of human aspect of software.  But I wonder, since you said you were a backend engineer, if you could talk to us a little bit about the suite of technologies that you're using to implement your product.  

IVAN:    Yeah, sure.  I work on the backend, which is a REST API, so we use Clojure, and we also use Datomic.  Datomic is our primary data store.  Everything goes into Datomic, essentially.  

We use a bunch of other tools that build the REST API.  We are pretty heavy users of Liberator, which allows us to define kind of restful controllers.  We have some kind of homegrown wrappers around Liberator, and we've also adopted a couple patterns about how we use Datomic in a kind of specific way that allows us to do certain things.  

One of the things that we kind of focused on--this goes back a little bit to when we were talking about usability--is often when you have a large number of people working on a bunch of different stories.  You have a bunch of different projects going on in your tool.  It becomes hard to see what's going on, so we spent a lot of time building this activity feed, which the goal was to try to make people be aware of and keep track of what stories they care about, and that actually was an interesting use case of Datomic as well because we had that transactional history there.  

CRAIG:    Hmm.

IVAN:    Definitely it's been an interesting experience.  I think I mentioned that earlier, which is, I've been programming for a while now, but this is my first job where I was using Clojure on a day-to-day experience, my day-to-day professionally, I guess, so that was definitely kind of a little bit of a learning curve there, and it was interesting.  It was a challenge for the last year, I guess.

CRAIG:    Yeah.  Let's actually go down that rabbit hole because I am always interested in people's kind of origin stories around Clojure.  You and I were talking a little bit before the show.  You started out in, I think you said, JavaScript was one of the languages, and so I'd love to hear your arch.  Where did you start out?  What's been hard?  What's been easy?  What's been surprising?  Those are all interesting aspects of your story to me. 

IVAN:    Yeah.  I guess it started out I did Python and JavaScript, although at the very beginning I wasn't very good at JavaScript, to be honest.  I didn't really understand it.  I was working at a company that no longer exists, but the primary application was running in Java.  And I was like, I'm never going to get into this.  This is just too much.

My friend and I, he was kind of a guy that was not super wild about Java as well, and he was also a Python developer.  We were thinking about, like, how are we going to build stuff.  He's like, I want to do everything in Common Lisp.  I'm like, what is Common Lisp?  What are you talking about?  I've never heard of this before.

Then I looked it up and I started learning a little about Lisp.   Then I was like, oh, I googled, basically, Lisp for Java, and Clojure was the first thing that came up, and that's kind of how I came to it.  This was like in 2008.  Then we kind of, he and I kind of, started learning about it.  We realized, well, this is really cool.  There are a lot of interesting ideas here.  It exposed me to a lot of things that I hadn't been exposed to before.  

Then eventually I moved on, and we never really used Clojure at that company, which was kind of funny.  It is still a Java shop to this day.  Eventually, I became a JavaScript programmer, and then finally, at this point, when I started working at Clubhouse, I started using Clojure day-to-day, so that was kind of my arch, I guess.

CRAIG:    Okay.  I'm always interested in what people have found to be good and bad, like what has been difficult for you, what has proven to be beneficial, especially the surprising things on both of those.  For instance, when I came to Clojure, and maybe this is similar to your experience, I was surprised by how much the functional mindset took to shift to.  That was the big challenge.  I wasn't expecting that.  What were your surprises and pluses and minuses?

IVAN:    That's kind of an interesting question.  I do remember one thing, which was when I was learning.  I don't actually have a CS background.  I came from more of a liberal arts background, and I've slowly learned.  Well, I guess I've learned some of CS, but there were definitely some things when I was learning about, like, JavaScript and Python, that were unexpected to me and that the functional mindset was actually a little more expected.  This was many moons ago when I think I was trying to see if two dictionaries were equal.  I called equals on them and it wasn't equal.  I'm like, wait a second.  These values are the same.  This is comparing by value or comparing by a point of reference type issue.  

CRAIG:    Mm-hmm.

IVAN:    Actually, some things about that when you kind of learn the functional mindset were actually easier.  Yeah, you're definitely right.  There are some things you have to get used to.  

I think one of the biggest things, well, I don't know.  There are a bunch of things.  The first was syntax because I had mostly a background in Python.  I was kind of like, oh, what are all these parentheses here.  I think that goes away pretty quickly if you're actually interested in learning about it because, like everyone says, you don't really see those any more.  You don't really care about those, and you learn Emacs, and the problem just goes away.  

There are definitely some things I think I remember learning, which is like you don't really have a for-loop, and a lot of algorithms, a lot of solutions in the world are kind of driven by these kind of complicated for-loops.  I remember I was really interested.  My background was kind of in mapping and chorographic stuff, and I'm very interested in computational geometry.  

If you tried to take a code, some piece of code, some algorithm that's written in Java that's kind of very specific about how we're going through this array or whatever and--I don't know if I'm making complete sense--translating it to a Clojure reduce, loop, or recur is not always the easiest thing.  One of the things, I guess, that's so much of the world has this kind of not even object oriented, but procedural, like you mutate things all the time world view that when you try to take that to  functional programming, there's just not as much even examples, really, or literature, I guess, about that.  I don't know if that makes sense.  

CRAIG:    Yeah.  Totally.  People on the show have talked before about impedance mismatch between the imperative mindset and the more functional, if you will, approach, and the higher level of abstraction.  I hope you won't mind if I tell a brief story.  I'll keep it quick.

IVAN:    No, of course.

CRAIG:    We used to have this thing that we would do internally that we called -- I forget what it was called exactly -- something cross-training.  But the idea was that we would pair up somebody from the technical side of the house, one of the developers, with somebody whose day-to-day job wasn't writing code, so maybe a project manager or whomever, and just have them talk about whatever.  Just have that kind of professional contact. 

One day I was paired up with our CFO, who is awesome.  She does not have a technical background.  She's managing our money, which she does a very good job of.  But her very first question to me was, "What is programming?" and I love that question - love that question.  Just a really smart question, I think, even though it seems so basic.  

One of the things I said to her was that it's about abstraction.  What is abstraction?  Abstraction is, if you're talking to a two-year-old -- you said you have a family, right?  If you were talking to a two-year-old--maybe you're familiar with this--and you want to go to the store, you say, "Okay.  Stand up.  Come over here.  Grab your shoes.  Bring them to me.  Now sit down.  Now give me this foot.  Now give me that foot."  Right?  It's very directive.  You're not really talking about the goal.  You're talking about the steps to achieve the goal.

IVAN:    Yep.

CRAIG:    Versus a more abstract conversation where, if I was talking to you, I'd say, "Hey.  Ivan, meet me at Safeway in half an hour."

IVAN:    Yep.  Yeah.

CRAIG:    Right?  That we would just go, okay, I know how to accomplish that.  It's really about the goal.  And so I think that there is a higher level of abstraction, which I believe is what you're referring to when you talk about having for-loops, which is, okay, do this, then this, then this, and then there's the aspect of mutation thrown in as well, which is kind of orthogonal, but versus do this operation to every element of this collection.  I find that to be a higher level of abstraction.

IVAN:    Yeah, totally, and I think it's a much nicer world to work in. Definitely.

CRAIG:    Yes.  Anyone that has a two-year-old knows that they are exhausting, and that's one of the reasons, right?  

IVAN:    You've got to bargain with them so much.

CRAIG:    Well, there's that too.

IVAN:    Yeah.  Yeah, you do have to go through all these kind of mundane, like kind of just simple, little steps.  You're like, okay, well, we've got to get your shoes on.  Then we've got to get your hat on.  And you're just like, no, I don't want to do this any more.

CRAIG:    Right.  Right.

IVAN:    Yeah.

CRAIG:    Yeah, maybe there's an analogy to error messages in there somewhere.  Anyway, okay, great.  Yeah, so you've been working with Clojure for a while now.  I think you said it's been about a year. 

IVAN:    Well, I've been interested in Clojure, and I've been playing with Clojure since, like, 2007, 2008.  But, yeah, doing Clojure every day has been, one, a great experience and, two, it's been about a year now.

CRAIG:    A year in.  You said it's been a great experience.  Now that you have got a pretty good amount of code under your belt, if you had to go work in another language, what things would you want to take with you?

IVAN:    Well, I think everyone knows the REPL is pretty awesome.  I really like Emacs.  I know some people in the community think that we should probably move beyond Emacs, and I agree with that, but I personally love it and I want to spend everything.  I do RFC in Emacs.  I want to do email.  I haven't figured out a good way to do that yet.  I really enjoy just working in Emacs and not ever having to leave it, and I actually find that as one of the nicest experiences, and you can do that in other languages, of course, but it works really well in Clojure.  

I find one of the things I like about it is it's a smaller community and there are not as many libraries.  Definitely Node has a huge library set, but I find that most of the time there's a library or there's some sort of kind of explanation on how to solve the problems that we have to solve at Clubhouse.  And most of the time it works pretty well, so I like the ecosystem around the tools, the libraries, and the community.  I think that's really nice and it's also nice that there are not, like, 500 different Web frameworks.  

I really like the kind of focus on building out frameworks around kind of protocols, I guess, like the Ring protocol, which are an API, I mean.  We don't have one monolithic framework, really.  We have a bunch of libraries that interact with Ring, and so I actually really enjoy that.  In the Python world, the kind of dominant Web framework has always been Django.  Django is really cool, and there are a lot of smart people who work on it, and I really like it, but there are definitely some things where it's not quite as appropriate or you don't want to use it for various reasons.  You can't just pull it apart and use pieces of it, and so one of the things I like about the Clojure world is you can pull things apart and use pieces here and there.  

Definitely I've been following along Clojure, and sometimes there's been this break where I feel like we're doing something one way and then there's this new way, and we have to kind of move to that new way.  I think that happens in all kind of programming languages and environments, so I'm not sure.  I guess I don't know what I'm saying there.

CRAIG:    I'm curious to get the same answer, but for a different technology.  I think a lot of our listeners have got some experience with Clojure.  I think many of them, but probably fewer have experience with Datomic.  Obviously you've built a real commercial product on top of it, as have I, and so I'm curious to get your take on really the same thing.  

A year in using this thing, what are your impressions?  What have you found beneficial?  If you had to go build something using some other technology, what would you miss?  

IVAN:    Yeah, totally.

CRAIG:    Of all that type of stuff, what's your arch with Datomic been?

IVAN:    I went through this experience.  I had been interested in Clojure, and I saw Datomic was released a couple years ago.  I was like, oh, I'm not sure if I understand what this is, so I really kind of took a fresh look at it a year ago.  I kind of went through the stage of not really getting it.  

    I've always used Postgres, and I've used Postgres very heavily.  And in fact, I've used PostGIS, which is like a spacial extension to Postgres, so I kind of understood that world a little bit, and so there really is this kind of -- I don't know.  I don't want to use paradigm shift, but there is a paradigm shift you have to go through.  

It started, which is like I was not really understanding how to correctly use it.  Then we definitely got -- my coworker Whit and I, we kind of really sat down and thought about a bunch of stuff about how we want to use it and tried to understand the Datomic way.  With a piece of technology, I think you want to understand what is its opinion about the world and try to not fight it, I guess.  I don't know if that makes complete sense.

CRAIG:    Mm-hmm.

IVAN:    It's not that Datomic changes.  It's like we had to change to Datomic a little bit.  Once we started to get it, we were like, oh, wow, this is really cool.  This allows us to do things that we could never hope to do. 

Then there was a stage where we're like, oh, and here are some things that we wish it didn't do.  Not necessarily, but it's more like the ramifications, like the immutable model enables you to do so much more than you could do with just a standard, like, SQL database, but we're starting to realize that there are some ramifications of that model that we hadn't quite thought about before that are definitely problems.  It's like a higher level problem in some ways.  It's problems you would never reach if you were using Postgres, but now that we have this complete history, we have to start to address them, which is kind of interesting.  I don't know if we've really solved them.  I think we understand some of them, but there's that.  

The biggest thing is I really like -- there's a bunch of stuff I like about Datomic.  The biggest thing is that the transactions as a data model is really nice and it's really good.  It's really nice to work with.  It's really nice.  

You know we get a bunch of data in the request, and it's from an endpoint that's creating some object in our system.  Then we validate it, we process it, we generate this transaction, and then we insert it into the database.  That has really worked really nicely for us.  The code is very enjoyable to write to do that process.  It's very easy to reason about, and I don't know.  It just feels like we're doing something right.  I don't know if that really makes sense.  Intuitively it seems like, okay, this is straightforward and there's not much mystery going on here.  

CRAIG:    Just to be clear.  You're referring specifically to the fact that the way that you put information into Datomic is via an API or entered around data and not around building some SQL string, right?

IVAN:    Exactly.  

CRAIG:    That's what you mean?

IVAN:    Yeah, exactly. 

CRAIG:    Okay.

IVAN:    Exactly.

CRAIG:    Okay.  Gotcha.

IVAN:    Yeah, yeah, exactly.  It's just like a vector that gets transacted and, in each vector, there are little pieces.  It ends up being like we have a bunch of optional.  We have optional data and we have defaults.  That ends up looking really nicely because you have a function that generates all those defaults or optional values for you.  It's very straightforward in a way that I never found something like Django's ORM to be very straightforward.  It doesn't look clear, I think.  You can also just take that function and run it, and not actually transact anything, and look at what's being generated.  It's very easy to see if that's correct or not.  

The other thing we've really enjoyed working with is basically that transact and query are separate, separate things, which we found pretty nice.  Query is obviously looking for stuff in the database and then transact is its own thing.  That actually has worked out really nice for us, and I think that makes our system a lot easier to reason about in a lot of ways.  

The other thing, which we started using, there was a talk that we saw about -- I forget the bank in Brazil that they've built a pretty sophisticated system in Datomic.

CRAIG:    Nubank.  

IVAN:    Nubank.  Thank you.  That talk was very influential about how we designed our system, so if people are looking, I think that was very helpful in myself's understanding about how we use Datomic because we started adding audit information into each transaction, which is really nice because we can now annotate transactions like who did it.  Not just who did it, but was this generated from a user or some sort of Web hook or something happening, another system, or what organization they were a part of, and all these other pieces of metadata that then later helps us process it.  Even like, is this transaction going to be of interest to a user later on and that type of stuff.  It's like I'm not sure.

It's nice because we can store that directly in Datomic and it fits very well.  We could, of course, do this in Postgres, I think, but it would be like an afterthought.  I don't know if that makes complete sense, but it would be something that would be kind of more external like this is: You look at the transactions.  You look at the audit information.  Okay, this is what's happened in this transaction, essentially.

CRAIG:    Mm-hmm.  Yeah, it's part of the model as opposed to part of your model.

IVAN:    Exactly.  Yeah.

CRAIG:    Yeah.

IVAN:    Yeah, so that's been really nice.  Datalog is not something I knew before using Datomic, and that took a little while to understand.  There are a couple things that we've realized that maybe we were using wrong, we were doing wrong.  

One of the interesting things is something that we did wrong originally is often in a Web framework you have, at the beginning of the request you open a transaction in whatever, say Postgres.  At the end of the request's lifecycle you close the transaction.  People do this different ways, but kind of like this is what happens in Django, perhaps.  You can do multiple inserts into that throughout that lifecycle of the request, and that's going to be in a single transaction.  

We were kind of originally doing multiple transacts in a single request, and I don't think that's how we were -- I'm not sure, actually, if that's how you're supposed to do it.  But we learned that that's kind of problematic.  We changed our system to try to group all the transactions or as much as we can.   We haven't been completely successful.  It's like a single request kind of generates most of the time a single transaction that inserts into the database, so that was something that we definitely had to learn, and that was not something that was obvious.  I think that comes from a traditional Postgres background or SQL background to moving to Datomic.

CRAIG:    Yeah, I know exactly what you mean.  You mentioned the separation of read and write as well, transact and query being separate.  That's actually something that I also have learned from Datomic and from Clojure itself, frankly, also separates--

IVAN:    Yeah.

CRAIG:    --updating things from reading things or, as Rich sometimes calls it, perception from permutation.

IVAN:    Yeah, exactly.  Yep.  

CRAIG:    I think it's a super, super powerful paradigm.  

IVAN:    Yeah.

CRAIG:    I wanted to call that out.  That's actually great.  You've mentioned a whole bunch of things that I also went through as I was learning various pieces of technology.  

I just want to make sure we revisit this because you said something really interesting, and I think you touched on it, which is you had to learn a bunch of stuff.  I'll paraphrase you as there were a bunch of ah-ha moments you had to go through.

IVAN:    Yep.

CRAIG:    I think you've mentioned a few of them.  You mentioned learning the fact that transactions are data.  That transactions are not a thing that spans a bunch of interactions with the database.

IVAN:    Exactly.

CRAIG:    But are themselves kind of atomic interactions with the database.

IVAN:    Yup.

CRAIG:    Were there other things that you can think of that people that haven't worked with Datomic, you can warn them and say, oh, you're going to have to shift your brain in this specific way?  You've mentioned a few, but I'm wondering if you can think of any others.  

IVAN:    Yeah, there are a couple more.  One of the interesting things is because Datalog is based on logic programming.  I'm probably going to say this incorrectly, but one of the interesting things that we discovered is that you build out these Datalog clauses in your query, and so you're looking for some piece of information, some record in your database, so to speak, and you're building out these clauses that are limiting.  Let's say it's the user object, so you say give me the users with this email and bind it to a question, the logic variable entity ID or something like that.  

What's interesting is if you miss, so we've discovered this a couple times in our queries that if you mistype entity ID on the left-hand side, if you accidentally type "EID" and you meant it to restrict your search, your query, what it's going to do is generate a new logic variable that's simple just not bound to anything.  I believe this is correct, and maybe I'm wrong.  That is kind of a subtle thing that it means that your query might actually return the correct results sometimes, but won't actually be correct.  

That's kind of an interesting thing that we discovered.  That's more about sloppiness on our part.  You have to pay attention to your Datalog queries in a way that, like, if you reference.  There's not a real good comparison in SQL, but if you try to select a field out of a table that doesn't exist, it will throw an error, for example.

CRAIG:    Right.

IVAN:    That was interesting.  I think one of the things that we've struggled with, and I don't actually have a correct answer for this, and I don't know if we've solved this completely, but Datomic stores a history of everything, but sometimes you have a bug.  Let's say, on your user object, you have updated that field.  That update, for some reason in your application, an updated field, updated at is like a timestamp or something else.  That doesn't get updated correctly for, like, three weeks.

What's interesting is that bug is now going to kind of live in the history of the database, which is fine.  Most of the time you're not going to care about that.  But once you start writing queries that go back and kind of are like auditing your information, you're going to have to take that into account, which is kind of like an interesting problem.  Whit and I have talked about … Whit and I have talked about how we want to solve this, and there are a bunch of solutions out there.  Well, I think there's one solution, which is, you could replay the transactions in Datomic, which would allow you to go back and basically fix the history.

The basic idea that I think we're trying to get at is sometimes you want to do a rebase, git rebase interactive and fix some of the transactions.  That's something that you kind of have to think about.  And the other thing, too, which we kind of didn't really consider as much because we weren't doing as must history stuff in the beginning is, as you add new properties to your entities, you need to take that into consideration as you're writing historical queries.  As you go back in time, if you're looking for some field on an entity and that field no longer exists, you're not going to get the results that you necessarily expect.

I think that's something that's also kind of similar to the rewriting history thing.  Kind of similar, but it's something that we haven't quite solved, but we've kind of learned to kind of work around, I guess.  Yeah, I don't know. 

CRAIG:    Yeah.  I think what you're pointing out is perfectly valid, which is that you're using a database, and one of the features of that database is immutable history.

IVAN:    Yeah.

CRAIG:    One of the things that's true about a database with immutable history is that history is immutable.

IVAN:    Yeah, exactly.  I'm not complaining about this.  I'm putting this out.  This is not a problem you ever have in a normal database system because you can never do this, so this is something that because Datomic is really interesting and it has this really cool way of storing data that we've been exposed to this, essentially.

CRAIG:    You can actually do this in a non-Datomic database, right?  This is what we mentioned before where it's no longer part of the model.  It's part of your model.

IVAN:    Yeah, exactly.

CRAIG:    You would have the same problems.  You actually mentioned, so we've run into this a bunch.  We've had other customers, other teams that we've worked with have said, oh, yeah, we have "bad" history.  I'm using air quotes around "bad."

IVAN:    Yeah, exactly.  

CRAIG:    We have things in our history that we wish--

IVAN:    Weren't there.

CRAIG:    Exactly.

IVAN:    Yeah, yeah.

CRAIG:    If only I'd known now what I know then.

IVAN:    Yeah.

CRAIG:    We have a couple answers to that, right?  One is, well, you did say it and, in fact, if you believe that one of the benefits is a record of what you actually said, well, that's what you actually said.

IVAN:    Yep.  Totally.  You've made this mistake, guys.

CRAIG:    Right.  Remembering your mistakes is good.  You mentioned, yourself, Git.  When we fix a bug, we don't try to necessarily go back and to get history and erase the commit where the bug occurred, right?

IVAN:    No, we replace it.

CRAIG:    That's right.  We override it.  We say, well, I got smarter and here is the new smarter.  But, you know, there certainly have been times where, for whatever reason, people have said we need to have a different history.  As you mentioned, it is possible to address that.  My colleague Stuart Sierra likes to call the process of taking all of the data out of the Datomic database and putting it into another database with an alternate history as decanting, which is a word that I kind of like.  

IVAN:    Like what you do with wine.

CRAIG:    Exactly.  Yeah.  It's the same idea, right?  We're going to leave the dregs behind.  Now, when we have a conversation with people about this, we're always careful to say, "Be sure this is what you want and you're not going this direction because of what you're used to.  Really consider whether the implications here are acceptable."  But it is an option at least.

IVAN:    Yeah, yeah.  No.  Sometimes it's for small things that don't really matter, and we don't want to do this for.  But, yeah, it's not something we run into like the application because the application mostly uses the state of the world as of right now.  

CRAIG:    Mm-hmm.

IVAN:    But sometimes it's more of like when we write custom queries that go look back at what did this organization -- like when was this organization created and how much did they do in their first week?  We have to think about that more.  Yeah.  

CRAIG:    Actually, I would love to talk a bit about specifics about how you're using Clojure, Datomic, whatever else you'd like to mention, in your product.  I imagine, given that you're trying to solve a nontrivial, real world problem, project management--

IVAN:    Yeah.

CRAIG:    --that you've had to come up with solutions to hard problems and that those solutions were interesting.  So I'm wondering if you wouldn't mind sharing one or two examples of things you ran up against and how you decided to model them.  Again, I think maybe the Datomic side of things is probably a little bit more interesting given the state of the world right now.  But whatever you think is cool, I'd love to hear a bit about your app.

IVAN:    Yeah.  Like I mentioned, one of the decisions we made -- this is another interesting thing about Datomic, I guess, which is, in Datomic, we have entities with attributes attached to them.  But the entities themselves don't really have a type.  In a normal SQL database -- this is another one that kind of, I think, paradigm shifts you have to go through when you're using Datomic.

 In your normal relational database you have a table called users, right?  But there really isn't quite the same equivalent--this is my understanding.  Correct me if I'm wrong, please--the equivalent of, like, a user's table.  You have a bunch of entities that have a bunch of attributes that resemble a user entity, essentially.  What's interesting is when we want to translate that to the front-end code, we have to kind of, in some ways, present that to them as if there is a user type.  Not exactly that, and this gets back to specifically what we were working on with the activity feed.  

What's interesting, we had to solve this problem, which is like there's a transaction and it's going to have a bunch of datoms.  Those datoms are going to represent a new entity and a bunch of facts about that entity.  But then we kind of want to say, okay, user Ivan created this story with this field.  One of the things that we did is we annotated the transactions with kind of a field that would allow us to quickly query for those in real time later on and say, "Okay.  In this transaction this object or objects were created," and that allowed us.  It made it simpler to present these transactions to the front end in a processed way that would look like something that you would recognize.

Some could say, well, maybe you shouldn't have done that, which is maybe true.  Maybe that wasn't the correct solution.  The front end uses kind of more of a standard MVC model, model controller view, world view I guess you would say, so we kind of wanted to present them, like this story was created, this epic was created, or this project, whatever.  That was one solution we did. 

That actually had some interesting ramifications about analytics, about you can see very broadly, very quickly, what people are doing.  One of the things we found was that looking, because you could go back, and you can say, for this time range, give me these transactions.  But loading the actual transactional data is actually not super fast, so we can write faster queries that use this field that tells us what were created and deleted.  It allows us to look at activity at a high level.

I don't know if we've solved hard problems.  We've also solved some mundane problems like we're pretty heavy users of schema.  And so we use Liberator to define our controllers, and we use Schema to validate our inputs.  That was something because we had originally kind of we were doing validation in different places kind of like we were doing transactions in different places.  I don't know if this is the right word for it, but we kind of moved our validation more to the edge and we did it more consistently, like if you give us an object that doesn't match what we expect you to--this is kind of normal stuff--we'll return a 400 and stuff.  

One of the more interesting things, though, and maybe this is my ignorance of kind of how restful APIs work, but we always have this kind of strange problem that Whit and I have to solve, which is if you're creating this object and there's a bunch of relational data in it because that's one of the cool things about Datomic.  It's relational.  We have projects, which are kind of like high level buckets of stories.  You try to create a story that's in a project that doesn't exist.  What should be the response code?

One of the interesting things is we've really tried to leverage Liberator to have that decision graph model to return error codes that are useful and learning Liberator's kind of decision model.  There's a lot in there, so we don't use all of it, of course, but it was kind of a big step that allowed us to build kind of a bunch of endpoints more quickly, essentially, and do it more consistently.  I think that was a big part of what we were working on.  

CRAIG:    Now do you find that you are making use of those fine grained HTTP status codes on the client to make different decisions?  In other words, are you distinguishing between a 401 and a 403 or whatever?  Are you actually using that to make smart decisions in the client?

IVAN:    Yes and no.  Yeah, we are using them in some places.  It's not complete, so I think the goal is one day we will use those to make smart decisions.  But one of the things that we're trying to do at Clubhouse is more than just find that balance between ease of use.  We also wanted to build a nice API that people could build stuff on top of our system without us having to ask.  

Whit and I have spent a lot of time trying to build an API that would actually tell you what you did wrong and help you, provide sensible error codes, tell you you gave me an int, and I expected a string, and so this is the problem.  This is the key that you screwed up on. 

I don't know.  I guess maybe we didn't have to do that, but that was something that we felt kind of strongly about.  We were having to do those checks anyway for our own consistency.  If you try to create a story with a project that doesn't exist, that's going to a null, and Datomic is going to explode or some other part is going to explode.  So we kind of wanted to make sure that there wasn't -- I don't know how to articulate exactly, but we wanted to make sure that all of that knowledge and those checks that we did were exposed to the front end or to any other client.

CRAIG:    Mm-hmm.  Yeah.  If I can put words in your mouth a little bit, you need to have something that programs can act on and not just a string that a developer can look at and maybe understand.  

IVAN:    Yeah, exactly.  

CRAIG:    Yeah.  Makes sense.  Cool.  I want to actually circle around to a different topic because you said something that was, to me, just absolutely fascinating.  I mentioned it briefly, this balance between usability and power.  I want to come back to that because I want to make sure we get to it.  I suspect it's something that you and your team have spent at least some time thinking about.  

First of all, I will go so far as to say you probably do not believe that it is a straight up tradeoff, that things that are more usable are necessarily less powerful and vice versa.  But I think that there is maybe something in there worth unpacking.  I'm curious to get your thoughts on that whole idea of usability and the idea of power and how they interrelate, if at all.

IVAN:    Uh-oh, now you've put me on the spot.

CRAIG:    You know.  It just seems like you've -- yeah, I am.  I'm curious to get your thoughts, so I am putting you on the spot, sir.  

IVAN:    Yeah, I think that's something that we struggle with a lot in tools and software, I think.  Right?  This is not an uncommon problem, and I don't know if I can answer this, so I get to duck out of the question a little bit.

CRAIG:    Of course.

IVAN:    I think it's a question of for who or whom.  I don't know how to speak English correctly.  I think we have different people we're trying to build our product for.  One of the things that we wanted to do was to make it, so most developers I know hate having to tell people what they've done and then put it into some system, right?  One of the things that we worked on a lot is to try to drive and record what happens in GitHub in Clubhouse so our interface, your interface as an engineer of interacting with Clubhouse and saying, "I'm working on this story and it's in development, it's ready for review, and then I deployed it," is through GitHub and Git, actually.  

That, we spent a bunch of work integrating with GitHub.  We do it based on branch naming, so if you create a branch with a certain pattern, the branch will automatically show up in Clubhouse.  Then actions you take with that branch will actually drive the story across the Kanban board or whatever.  

I don't know if that answers partly one of your questions.  For engineers, for example, I think that makes Clubhouse easier to use.  I don't know.  

The other thing is I think, with a lot of project management tools that I've observed that you can do a lot, it's hard because you can do a lot of different things with it, but it ends up being a little unwieldy.  What we've tried -- I don't know how to answer this exactly.  What we tried to do is--

CRAIG:    You've brought up this idea of introducing features that are aimed at smoothing a road, right?  You know that a lot of people are going to choose to have branches.  Those branches are going to be in GitHub.  They're going to take certain avenues.

IVAN:    Yep.

CRAIG:    Or take certain actions on those branches.  You're like, oh, well, let's use that knowledge and use it to do work that they are going to do anyway.

IVAN:    Exactly.

CRAIG:    To move a story through a workflow.  I guess the question I have for you is then, okay, given a feature like that where you've kind of decided to pave the cow path, is not quite the right analogy, but I think you see what I'm getting at.    Do we give up anything by doing that?  We'll go ahead and pick on the 800-pound gorilla, JIRA, right?

IVAN:    Yeah.  I wasn't going to say it.

CRAIG:    Yeah, I know, and you rightly should not.  I think it shows a great amount of taste and good behavior when people speak about facts rather than picking on their competitors.  You haven't done it, but I will for you.  I'll take JIRA as an example where you can get it to do anything.  But as a result, I might have to do a bunch more dance steps.  That's kind of the tradeoff I'm talking about. 

IVAN:    That's the idea with the GitHub integration.  I guess this is just one example, our approach, I think.

CRAIG:    Sure.

IVAN:    As you say, people are doing this anyway, and you are moving things into different states.  The idea of having some tool outside of your version control system that manages these stories is that other people can have insight into what you're doing, like sales and product, and everyone can see.  Okay, Ivan is doing this and working on that.  

I think the nice part about this is that it still does that, but I don't really have to do anything different in my day-to-day life as an engineer to get those stories to move across and other people can see them move across, and everyone can see, like, oh, hey, look at this.  This story, which was estimated at 2 points, was 5,000 lines of divs, so maybe it was more than a 2-point story.  You know that type.  

I think definitely I don't know if we're quite there, but essentially we can start doing some stuff with that information, like saying, hey, this story is huge and it's five branches.  Maybe this needs to be broken into multiple stories.  Obviously we're not there yet, but we're starting to get the information so we can kind of get insight into that at least. 

CRAIG:    Yeah, it's interesting.  I think what I'm taking away from this is that, no surprise, once again we've discovered that by taking the problem apart and focusing on what are you really trying to do, that there's wind in that.

IVAN:    Yeah.

CRAIG:    I'm hearing you say, well, looking at it, you're not actually trying to move a card from column A to column B.  That's not the goal.  The goal is to provide visibility into what's going on.

IVAN:    Exactly.

CRAIG:    And so if we can find a way to do that, it doesn't really matter if it's a developer dragging a card between columns or committing to a branch.  And if they're going to commit to the branch anyway, then let's not make them do other work that's equivalent given the goal that we've thought about underlying the real problem.

IVAN:    Yeah, exactly.  There's a whole bunch of other stuff we've done too.  I think Andrew Childs is the front, kind of like the lead guy on the front end, and he's done other things, which work really hard to make a tool with a lot of knobs and levers where you can configure things, you know, like make it look nice and make it so it's easy to find all those different pieces.  In some of these tools a huge problem is that there are places where stuff goes to hide, kind of.  I don't know if that makes complete sense.  That's a slightly different issue.  

Yeah, that's part of the idea of building an API that's public that people can build more tools into.  We haven't quite got to the deploy.  

Oh, someone is at my door.  I'm sorry.

CRAIG:    All right, we're back.  Had to take a quick break there for a second, but Ivan, you were saying -- please continue.

IVAN:    Yeah, I think that's one of the reasons why Whit and I focused on building an API where anyone could kind of build integrations to.  Part of the story that we haven't completed yet is that when you finish a story, you open a pull request.  You close the pull request.  It gets moved into done, and then the story gets deployed.  You want to have that happen automatically, so we’re hoping that--and we're going to work on this at some point too--that story can travel across the Kanban board the whole way without having to have -- you don't actually have to drag it there because you're already taking these actions.  We just need to build the integrations to drive those.  That's nice because then you have a record of that associated with the story.  The story explains in written English what you're trying to accomplish.  

Rather than, oh, I finished this and I forgot to move it into the done state for a week, you have a very accurate history of, like, okay, this is when we started it.  These were the commits that started the story.  This is when they merged that pull request, and then this is when they deployed it.  I think that's one of many things we're trying to do.

CRAIG:    That sounds very cool.

IVAN:    I think it is.

CRAIG:    No, like I say, I have not used the product.  But after talking to you and, of course, I'm a big tech fan.  I know that you're using neat technologies in the backend.  It makes me want to use it more.  I think, too, my take on things is that this sort of thing is really needed.  Anyway, we could go there, but I think maybe we'll save that for some other time. 

I want to make sure that, since you're a busy person, you're trying to develop this product, I don't want to keep you too long, but I want to make sure that we clear any space that we need to here to talk about anything else that you'd like to discuss today.  If you don't have anything right now, then that's cool.  I think we should have you back on again in a while, see how it's gone, see how your knowledge of Clojure and Datomic has continued to expand.  I'm always fascinated to hear how people have come to understand these technologies better.  But like I said, that's for another time.

This time, is there anything else you'd like to talk about today before we start to wrap it up?

IVAN:    I guess not really.  I guess maybe I have to say thank you to the Clojure community because I've been in it in various forms for a while now, and it's kind of like a nice group of intelligent people.  And I've really enjoyed learning a lot.  I mean I feel like my journey with Clojure has been kind of like understanding so much about even like computer science on some level, and so I think that's been really interesting and it's been nice.  It's a nice language, and it's a nice group of people, it seems like, so I really appreciate that, actually.

CRAIG:    I totally agree with you.  I've had the same experience.  It is really wonderful.  I mean you know it's hard.  It's sort of an amorphous Internet based thing.  

IVAN:    Which is true, yeah.

CRAIG:    At the same time, there are an extremely large number of people who are just awesome, and I've learned a lot too.  It's good stuff.

IVAN:    Yeah.  

CRAIG:    Well, cool.  Well, Ivan, it's been really interesting to talk to you, just so fascinating to me to kind of see how things have gone.  In the old days -- gosh, this podcast has been going for five years now, and I don't think I've ever known everyone that was doing Clojure.  I haven't been in it that long.  I've been in it since roughly around the same time you were when it was a pretty small thing, but there was never a point where I knew absolutely everyone that was doing Clojure in the world.  

But it is definitely the case that, over the five years we've been doing this podcast, the number of times where we say, "Oh, we should talk to that person," and they're doing something that I just hadn't been aware of, has gone way up.  I think that's because there's just a lot of people now doing interesting work using these technologies, yourself included.  Always super fun for me to hear about one more of them, and that was no exception today.  I will thank you a ton for coming on.

But of course, before we really sign off, we're going to ask you one more question.  This is the question about advice.  We always ask our guest to share a little bit of advice.  That advice could be technical.  It could be non-technical, which it often is.  It can really be anything.  It's any kind of piece of advice you'd like to share with us.  What would you like to share with our audience today?

IVAN:    I'm not sure.  I think, for me, Clojure has been, like I just mentioned, kind of down the rabbit hole, and I've kind of not held back, and that's been a really great experience for me.  Definitely when I first started, there were so many foreign concepts, so many words I had to look up and just try to understand.  I kind of never looked back, and I just kind of let it kind of take over what I was obsessed with.  I don't know.  It's been kind of fun.  I don't know if that's really advice, to be honest.

CRAIG:    Be like Ivan in that regard, I think, is good advice.  No, I mean embrace it, really, like follow your obsession.  Go ahead and see how far you can take it, I think is fantastic advice.

IVAN:    Yeah, and that goes to what I think my friend Rob once told me, which is like, you know, we get in these arguments about what's best and what's the best way to program.  Not in this community necessarily, but definitely on the Internet.  My friend Rob was always like, "You know, just let people do what they want because they're going to get obsessed with it, and they're going to be, like, really into it and really kind of care about it.  If you force them to do something, learn, use some language or some toolset that they don't like, they're kind of going to do it begrudgingly."  

I don't know how to make programmers do that.  I think programming is such a mentally intense thing that you really have to kind of embrace it.  That's what I found, personally.  I don't know.

CRAIG:    Mm-hmm.  Yeah, I think you're right.  I mean I think one of the things that a lot of people talk about when they talk about hiring and when they talk about the teams they want to work on, the word that comes up over and over again is passion.

IVAN:    Yeah.

CRAIG:    How do you make somebody passionate for something they can't stand?

IVAN:    Yeah, you can't.

CRAIG:    You can't do it.

IVAN:    I don't know if that's a good thing, actually.  I'm not sure.  Is it great that we have to be passionate about it?  But it feels like we do.  I don't know.

CRAIG:    Yeah, and I know what you mean, but it's certainly true, and so learning to work within that constraint is with wisdom, right?

IVAN:    Yeah, I guess so.

CRAIG:    Yeah.

IVAN:    Yeah.  All right.

CRAIG:    Cool.  Well, Ivan, thank you so much for taking the time to come on and talk to us today about your experience with Clojure and Datomic and about the interesting work you're doing at Clubhouse.  I really wish you a ton of luck.  I mean it looks like you've made a great start.

What I'm really looking forward to is a project where I get assigned to the project and they say, oh, our project management tool is Clubhouse.  And I'll be like, oh, yeah, great.  It hasn't happened yet, but I'm looking forward to the day sometime soon when that will happen, and I'll be like, I know that guy.  This is good stuff.
    
But anyway, certainly thank you for coming on and talking to us today.  It's been really fun having that conversation with you, so appreciate it.

IVAN:    Yeah, thanks for having me on.  I appreciate it.

CRAIG:    No, not at all.  We'll have you back some time before too terribly long.  But for now, we will go ahead and sign off.  This has been The Cognicast.  

[Music: "Thumbs Up (for Rock N' Roll)" by Kill the Noise and Feed Me]

CRAIG:    You have been listening to The Cognicast.  The Cognicast is a production of Cognitect, Inc.  Cognitect are the makers of Datomic, and we provide consulting services around it, Clojure, and a host of other technologies to businesses ranging from the smallest startups to the Fortune 50.  You can find us on the Web at cognitect.com and on Twitter, @Cognitect.  You can subscribe to The Cognicast, listen to past episodes, and view cover art, show notes, and episode transcripts at our home on the Web, cognitect.com/podcast.  You can contact the show by tweeting @Cognicast or by emailing us at podcast@cognitect.com.  

Our guest today was Ivan Willig, on Twitter @ControlCMetaP.  Episode cover art is by Michael Parenteau, audio production by Russ Olsen and Daemian Mack.  The Cognicast is produced by Kim Foster.  Our theme music is Thumbs Up (for Rock N' Roll) by Kill the Noise with Feed Me.  I'm your host, Craig Andera.  Thanks for listening.