In this episode, we talk to Creighton Kirkendall about Clojurescript, React and Om.
In this episode, we talk with Creighton Kirkendall about Clojurescript, React and Om.
RUSS: Hello and welcome to Episode 119 of The Cognicast, a podcast by Cognitect, Inc. about software and the people who create it. This week marks the second outing of Carin Meier as the Cognicast host. This week Carin is talking to Creighton Kirkendall.
But before we get started, we do have a few events to mention. First, there’s a ClojureBridge happening in Boston on March 17th. In case you don’t know, ClojureBridge is dedicated to increasing diversity within the programming community by offering free, beginner friendly Clojure programming workshops to people from underrepresented groups. I can tell you from personal experience, ClojureBridge workshops are a whole lot of fun as well.
There’s also the clojureD conference happening on February 25th in Berlin, Germany. ClojureD is an independent, nonprofit conference from the Clojure community for the Clojure community.
If you have a Clojure related event you would like us to mention, please drop us a line at podcast@cognitect.com.
Well, that’s about it, so on to Carin and Creighton in Episode 119 of The Cognicast.
[Music: "Thumbs Up (for Rock N' Roll)" by Kill the Noise and Feed Me]
CARIN: Are you ready?
CREIGHTON: Yep.
CARIN: Okay. Well, welcome, everyone. Today is January 27th and this is The Cognicast. I’m Carin Meier and today it’s my great pleasure to welcome Creighton Kirkendall to the show. Thank you for being with us, Creighton.
CREIGHTON: Cool. Thank you very much for having me.
CARIN: Yeah. I’m very excited to have you here on the show, especially since you’re from my hometown of Cincinnati, Ohio, with me, so that’s kind of exciting.
CREIGHTON: Yes, we are definitely evangelists for Clojure here in Cincy, so I’m excited to do this.
CARIN: Cool. So as we begin every Cognicast, we start out with an art question, so I’d like to ask you if you would like to relate some experience of art, whatever that means to you.
CREIGHTON: Well, it’s interesting. I actually got into computers because I could draw. I never intended to be a programmer.
CARIN: Cool.
CREIGHTON: I actually wanted to minor in art and be a chemistry major. I was actually working at Career Services as a freshman at Wright State University, and they found out I could draw, and I actually did portraits for people and used to give them as gifts. So one day I was filing papers, and they actually had a database report they had asked for. At this point I didn’t own a computer and I didn’t even know anything about computers.
CARIN: Wow.
CREIGHTON: The most I’d ever used them was to actually draw on them. I used to do, like, artwork, like cartoons, but I would do them one pixel at a time in paint. Oh, like the idea of doing that now just absolutely is crazy, but I used to love it. I used to do it all the time.
They found out I could draw, and they got this database report. They were arguing over it, and the network engineer was trying to help them. I walked over, and I looked at it. I said, this is what I would ask for if I was trying to get what you’re looking for.
Then the next day the network engineer came up to me and asked if I wanted to be his assistant and to help them build a website because he knew I could draw and he knew that I had some technical aptitude. At the time, I had none. Zero.
CARIN: Wow.
CREIGHTON: But I was like, sure. Why not? Because I was good at math and stuff like that, so he was like, you could do this. No problem.
Things got a blur after that. I actually stayed as – I was a comprehensive education and chemistry major, and I stayed in that for the next year. But by the end of that year I was actually over at NCR writing software for them.
CARIN: Oh, wow.
CREIGHTON: I ended up switching majors into math because I liked it a lot better, and eventually I kept the math degree and eventually went on to get a computer science degree. But it really all started filing papers in Career Services at Wright State.
CARIN: I just love hearing stories about people’s paths to where they are because it’s not always a straight road. It’s kind of windy.
CREIGHTON: No, it was crazy because I got in at NCR with a great group of people. I mean they were phenomenal, and they taught me an enormous amount and basically let me do way more than anybody should have ever let me do. Oh, my gosh. If I think back to the code I wrote, oh, I just want to crawl under the desk.
I can’t believe they let me do it, but they taught me a lot, like one of the guys I worked with was actually part of the team that did ARPANET. Oh, and just the discussions that we would have. To this day they’re some of the greatest learning experiences I had. NCR had some rough times and stuff, and they actually – one of the guys there got me on at a startup down in Cincinnati, and that’s how I ended up relocating down here.
CARIN: Oh, wow. I wasn’t aware. Yeah, for some of the listeners, they might not be familiar with NCR.
CREIGHTON: Yes, NCR, they used to create ATMs, and they actually developed computers for a while. They were a big competitor of IBM back in the heyday.
CARIN: Ah, okay.
CREIGHTON: It’s National Cash Register is what–
CARIN: Oh, that’s right.
CREIGHTON: All your point of sale devices and stuff, you’ll see a little NCR tag on a lot of it.
CARIN: Yeah. We’ve known each other for a while now, but when we first met I was amazed by your art ability. I think it might be a little known fact to the listeners out there that you are responsible for the 4clojure logo.
CREIGHTON: Oh, my goodness. I knew that was going to come up. The 4clojure dragon.
CARIN: Yeah. Yeah.
CREIGHTON: Yeah, the dragon logo. I have to kind of chuckle. I actually did a ton of logos for him. I probably did–I don’t know–probably 15 different logos, and the dragon was done as a joke. Like at the end I sent, I think it was David–
CARIN: Mm-hmm.
CREIGHTON: I sent him a note that says, “Here’s one. I figure everybody likes dragons and pirates,” and I sent that over, and that was the one they picked.
CARIN: I thought it was cool.
CREIGHTON: I would have never have guessed. But there were a lot more, like I was trying to be all professional, and the reality was that fit that site way more, way better, and the people involved in it.
CARIN: Yeah. Yeah, it’s awesome, so you’ve got a little known fact.
CREIGHTON: Hmm.
CARIN: Yeah, so Clojure, just a little bit about – we’ve talked about how you’ve gotten into programming, but what about Clojure specifically? What were you doing before you got into Clojure, and how did that happen?
CREIGHTON: I’ve always been a bit of a language nerd. I started out in Pearl, then did a lot of VB.NET at NCR and even some Cold Fusion. But I got hooked into Java near my tail end at NCR, and built Java systems for the next, like, 16 years I was an architect in Java. I enjoyed that. I did, but I was playing with languages all the time because I even wrote a few languages just messing around.
CARIN: We’re going to come back to that because I want to know.
CREIGHTON: But I kept coming back to, like if you look at Enterprise Java and when you’re dealing with large systems, there’s this idea of trying to figure out how to make things composable and so that you can plug and play, and basically trying to create these Lego blocks. But it’s fairly challenging and there’s a lot of stuff that has to go in it at that in the … world.
I started hearing more and more about these functional programming and, in particular, about a language called Scala. I was like, wow, I need to learn this functional programming because it seems to solve a lot of the problems that I am trying to get architects and stuff to understand. It seems to know, so I started diving into Scala. I wrote a system in Scala and it turned out I went and looked at other people’s code in Scala and I couldn’t even read it. It was because I thought I knew Scala.
CARIN: Mm-hmm.
CREIGHTON: What I didn’t know was how to program functionally.
CARIN: Hmm.
CREIGHTON: So I didn’t know anything about it. I was like – I was basically taking my knowledge from Java and trying to write Scala, and I realized that to write it correctly there was a whole other world under the covers.
CARIN: Mm-hmm.
CREIGHTON: The way that they were writing was a lot more composable. Like they could – like things just sort of flowed a lot better. And I tried to write that way, but I kept falling back on my old things because it was easy to do.
One day I stumbled across a Lisp written for the JVM, and I’m like, well, that would force me to do it. That would force me to do it. I wouldn’t – there’s no going back at that point. I’m like I’m in Lisp world.
I started looking at Clojure. At first I did not realize that Clojure was like a full language that was going to take over things like it did. I was just sort of – I wanted something to play with. I actually went through the first three lectures of the Berkeley’s Introduction to Programming because they’re done in Scheme.
CARIN: Hmm. Okay.
CREIGHTON: That is actually how – I went to that so I could read Clojure code. Then from that I actually ended up – my very first Clojure program was A Part-Of-Speech Tagger that eventually became a – it grew into a syntax checker. Not a syntax, but a grammar checker. Yeah.
CARIN: Nice. Part-Of-Speech, my English background, you know, from school isn’t great, so that would be like nouns and verbs and looking at sentences? Is that what that was?
CREIGHTON: Yes. It basically – I can – I am horrible at English. I was diagnosed at five with dyslexia.
CARIN: Oh, okay.
CREIGHTON: I am really bad at sort of the language side of things. I’m fairly good now, but the startup I worked for was a lot of English majors and communications majors, so I learned a lot on the job to try to remove the red that came back on some of the things that I used to…. Part of why I picked that first is I had tried to write it in Java. I tried to write A Part-Of-Speech Tagger and a grammar checker in Java that would catch the problems I had because dyslexia has some interesting things that it doesn’t just – it’s not that you just misspell things.
It turns out that the types of problems that you have, they’re very uniform and they’re fairly common amongst people with dyslexia. Like endings of words are a struggle. For some reason I don’t see them. I don’t type them. Like -ing, -end, -es, and those, when I’m typing they don’t come out.
CARIN: Oh. Huh.
CREIGHTON: They just don’t, and I don’t see them when I read them. Why, I don’t know. I haven’t figured that out. But after talking to a few other people that have dyslexia, it turns out that’s a very common thing.
CARIN: Hmm.
CREIGHTON: Things like a context sensitive spell checker that would actually spell check, but in context, so it knew that it was a verb or it knew what it was. The first thing I had to do was write the Part-Of-Speech Tagger to be able to pull that off. I tried to write it in Java, but it’s not a trivial thing to try to write in Java.
CARIN: No. I think there are whole fields of study devoted to this….
CREIGHTON: But in Clojure, for some reason Clojure worked in my brain. Part of that is I definitely lean mathematical, and I loved Clojure because it was like the shortest distance between the way my brain worked and getting the stuff on the screen, like typing. I sort of worked out the algorithm for how to actually do it and, over the course of I think four days, I wrote A Part-Of-Speech Tagger in Clojure. It was 89 lines of code.
CARIN: Wow.
CREIGHTON: I was so proud of it. Now it was written in Scheme format because I didn’t really know Clojure at the time. I only knew Scheme from the three lectures. Everything was structured a little weird, and I’ve cleaned it up since. I actually did a talk recently, like an intro to statistical learning where I actually used it as the basis for the talk. But I ended up building a cool, little grammar checker, and the grammar checker even works in ClojureScript.
CARIN: Nice.
CREIGHTON: It exports out, and I do use it.
CARIN: Nice.
CREIGHTON: Which is…. I do cut and paste a lot of my emails over into it to make sure that I don’t screw up and haven’t done horrible things. Now Google has gotten a lot better. Google’s checker is probably the most, biggest godsend to someone like me because every time I go to type in any field, it highlights when it thinks that it’s wrong.
CARIN: Yeah. I appreciate that too. Definitely. I didn’t want to lose a thought because when you were saying that I went, oh, I want to come back to that. You said you wrote some languages.
CREIGHTON: Oh, yes. The very first open source stuff I did was actually a tiny, little math language where it was for me and a guy out of – he was out of Melbourne, Australia. He was actually from mainland China, but was studying at the University of Melbourne. He was in bioinformatics, and he wanted to basically write something that would allow him to do statistical stuff.
What he was looking for was NumPy. That’s what he was looking for. At the time, this was in 1999.
CARIN: Uh-huh.
CREIGHTON: It was prior to a lot, and I didn’t know Lisp. I didn’t know a lot of things, so what I wrote was something that looked a lot like Java, operated slightly different with a little bit of MATLAB and Mathematica thrown in to try to make it a little easier to deal with things, but it was horrible. It was just a horrible language. And that’s when he discovered Python.
CARIN: Ah, okay.
CREIGHTON: We both agreed that this project should probably die because that was way better.
CARIN: Did you learn some lessons from writing that?
CREIGHTON: Yeah. I did. I learned a ton of things. In fact, from that day forward how I actually learned new languages was implementing Lisp and stuff in them, like trying to implement a little language, like just a little scripting language.
CARIN: Mm-hmm.
CREIGHTON: Eventually, after I’d learned Clojure, I started implementing Lisp in every language. I don’t know why. There’s just something really cool about when you go to implement a Lisp. No one realizes that you can do it with just a tiny, tiny subset of functions. There’s only like six special forms and this tiny, tiny, little piece of code to do to implement a Lisp, and it’s a thing of beauty.
CARIN: Yeah, Lisps, definitely that is one of the qualities I love about it is the simplicity. Yeah. We talked about how you got into Clojure. Since then you’ve actually been doing quite a bit of open source work. You have a few projects out there that are very useful and popular. In particular, I think there is – is it “Q”? How do you pronounce it? I’m never sure. It’s K-I-O-O.
CREIGHTON: I say “key-o”.
CARIN: “Key-o”, okay.
CREIGHTON: However, I have been told that that’s not really how you’re supposed to pronounce it, but that’s how I pronounce it. But it’s basically Swahili for mirror.
CARIN: Ah, okay.
CREIGHTON: But the idea is that it is taking – it is allowing your data to basically be transformed into HTML, so it’s like the mirror of the data, a different transformation, a different view of the data. That was the idea behind it.
CARIN: How did that come about?
CREIGHTON: Well, that started out – Kioo came out more as a reaction to me finding React and some discussions with David Nolen and finding out, but it’s responsive because I wrote it in Enfocus, and Enfocus has the same idea, but it’s not React based. It is more of a traditional sort of model, a sort of DOM manipulation versus React’s model, which is the whole DOM diff style.
CARIN: Mm-hmm.
CREIGHTON: But they’re very similar in how you use them, like Enfocus, and I actually use Enfocus for all my testing because I actually render the stuff using React and then test it using Enfocus.
CARIN: Oh, interesting.
CREIGHTON: Enfocus, the idea is that I allow a designer to design static HTML, like a walkthrough, but then I can actually take that and directly bring it in and transform it using something similar to Enlive, which for those that are familiar with Enlive, if you think about it it’s a transform language that was designed by Christophe Grand that allows very similar things like XSLT is probably the closest thing you can think of to a transform language, but this is a programming transform language. It’s much easier to use. It uses basically CS3 selectors or XPath selectors that allow you to basically compose transforms.
But I did this actually as a response to – I was giving a talk on Clojure and introducing ClojureScript the very first time I introduced ClojureScript, and I was in a room full of Java developers. One of them asked a simple question, “How do I integrate a designer into this?” because what I was showing was Hiccup.
CARIN: Hmm.
CREIGHTON: Developer productivity has always been a big focus of mine, so when I looked at it, I’m like, really you need to be able to sort of let the designer do what they want to do and let the developer do what they’re good at.
CARIN: Mm-hmm.
CREIGHTON: What I want to do is basically take what the designer has and basically put my data into it the way I would expect to.
CARIN: The workflow then would be the designer can just make their mockups in like whatever they’re comfortable with?
CREIGHTON: Mm-hmm.
CARIN: And then the developer takes that and then can transform those static pages into the–?
CREIGHTON: Yeah, they actually get – they get compiled and brought in and, in essence, it compiles the transform so that it’ll go get the CSS3 selectors, find the little bits and pieces, and replace them–
CARIN: Oh, okay.
CREIGHTON: –in real time. Now what’s interesting is I spent quite a bit of time thinking about this problem, and I was actually sitting at Clojure/conj back in–was it–2010.
CARIN: Time flies.
CREIGHTON: It was 2010 or ’11. I was sitting at Clojure/conj and I was sitting next to, I think, Sean Corfield was sitting next to me, and we were actually discussing Cold Fusion because I had done Cold Fusion. But he showed me his – he was building a Web framework that was similar to the Cold Fusion framework for Clojure, and he showed me. But what I saw out of that, he introduced me to Enlive.
CARIN: Hmm. Mm-hmm.
CREIGHTON: I was looking at it, and I’m like, that’s what I need. I need that in ClojureScript. That’s what I need. I need that. That’s exactly what I’m looking for. Actually, I’m trying to remember his name, but he was actually from Neo Columbus.
CARIN: Okay.
CREIGHTON: After that we were discussing how to actually implement that in ClojureScript with them, and we both went back and forth on a couple different things. He said him and another guy had actually attempted it. After that, I was off between Christmas and New Years, and I sat down and wrote Enfocus.
CARIN: Nice.
CREIGHTON: Between Christmas and New Years, and it just worked. In fact, it was probably, like that was one of the things where I realized just how important ClojureScript could be because it was the beginning of understanding that ClojureScript allowed you to build abstractions that just weren’t possible before. Like the compile time macros and stuff like that allowed you to do things that no other language has, in essence. It made it so much easier for me to build sites and stuff.
CARIN: Mm-hmm.
CREIGHTON: So I started doing all kinds of things in ClojureScript. Then React came along, and I wanted the same thing, so Kioo was born, and Kioo was a lot harder because that’s a full compiler. It was a lot more difficult to build, but it turned out really nice.
CARIN: You used your kind of experience with Enfocus and then you were able to build out Kioo.
CREIGHTON: Mm-hmm.
CARIN: Nice.
CREIGHTON: Yeah.
CARIN: I’ve used it on a couple projects, and it is very nice, so I appreciate it.
CREIGHTON: That’s good.
CARIN: You’ve also been busy doing some other things lately, too. I’ve noticed on your GitHub there’s a project that has to do with some games.
CREIGHTON: Peli. Peli is a game engine. And what’s on master is just a very simple game engine that allows you to build little Mario-style games, and there’s very rudimentary physics. But there is a branch that actually has a full immutable 2D physics engine built into it.
CARIN: Whoa!
CREIGHTON: It now runs in both Clojure, ClojureScript, and hopefully ClojureCLR here soon.
CARIN: Wow!
CREIGHTON: It actually is a full 2D physics. It’s a full engine. I can build pretty much any game in it at this point. There’s some basic things that I want more sort of things like springs. I want to add springs in, and I started to, but I haven’t got the time because there’s a lot of things that have been taking my free time recently. I coach wrestling.
CARIN: Oh, goodness.
CREIGHTON: Yeah, so I coached soccer and wrestling for years, so they definitely consume some of my time when I’m not programming.
CARIN: Sure. I know I’ve worked a little tiny bit with the games industry, and I know that speed is really important in games and game engines. You were talking about immutability. How did you balance the two and have you been able to get good performance with the immutability?
CREIGHTON: That was definitely the big challenge. There were certain things I did at first like I built the immutable thing, but I couldn’t get great performance. I actually used Core Matrix to build it.
CARIN: Mm-hmm.
CREIGHTON: Because it now worked in ClojureScript. I’m like, sure, we’re going to do that. But it turns out that the type checking and stuff slowed things down. But once I actually sort of worked that out and sort of optimized a few things, there’s still a lot that can be optimized, but I get well more than 60 frames a second.
CARIN: Wow.
CREIGHTON: It turns out that the JavaScript virtual machines like JavaScript Core and V8 have advanced to the point where they’re stripping out a lot of these 2D physics engines that are written in Java that are highly optimized. They’re really, really fast now. This one, it’s not as fast, but it is close to most of them. It’s within maybe two times the speed of most of the engines that I’ve tested it with.
CARIN: Nice. What advantage then does the immutability give you in the games engine?
CREIGHTON: Most of it is around how to debug. I can have a time travel debugger, and that’s really the thing with Peli that why I had to write my own 2D physics engine because that sounds crazy and for the most part it is. For math nerds it’s like, oh, that’s awesome. I’m a math nerd, so I was having fun with it. To write one it’s not a trivial task, and it takes a fair amount of time. I did it because I wanted my time travel debugger, which I had written for the original Peli, which allows me to basically run the game, and I can stop the game at any point and back up and fix bugs, and then press play and it’ll continue on as if from the point that I backed up to only with the new code.
CARIN: Nice.
CREIGHTON: I basically can do live editing and actually back up, so if a collision doesn’t do what I want, let’s say Mario goes jumps on a goomba or whatever, and he doesn’t squash, that’s bad. We back it up, and I can fix the code. Then press play, and all of a sudden he squashes the goomba. That’s really fun. That’s why I actually built the 2D so that I could basically still have that ability, but have some much more sophisticated physics, so I could do a lot more different types of games.
CARIN: What sorts of games are you interested in?
CREIGHTON: I’ve got – I mean I’ve got a bunch of little ideas that I play around with. Some of them are just like one idea is where, if you think about it, it’s just a ball traveling in space, but it has to stay in contact with the wall to get power. If it runs out of power, you die. But here’s the catch. The catch is it’s a magnet, and you get to decide which floor is magnetized.
CARIN: Oh.
CREIGHTON: And if it hits certain things, it gets demagnetized and you can lose power and stuff. That’s kind of cool. There are some other network games like I have a game called Flew Fours (phonetic) that my kids want me very badly to build that’s much more complicated, but man would it be fun. It’s going to have poof ball type characters. They’ve got to battle. They’re a little bit cannibalistic, so they’re going to … those types of things.
CARIN: Very cute, a cannibalistic kids game.
CREIGHTON: Yeah.
CARIN: Of course.
CREIGHTON: Exactly. You can see why my kids would love it. They’re like, “Yay! Cannibalistic cute things. Sure.”
CARIN: Oh, goodness. Yeah, there’s all sorts of games out there right now that are just crazy with their graphics and also now just with like the artificial intelligence behind them. That’s kind of a crazy component of games. I think that’s some aspect that you’re interested in as well, right?
CREIGHTON: Yes. I definitely have been diving deep into the AI for about the last nine months. A lot of it is driven, again, because of the sort of natural language stuff that I did a while back with the A Part-Of-Speech and stuff, so I started in that. But the more I’ve gotten into the deep learning stuff, the more game AI, all of that.
I’m beginning to understand how it all works. It took me forever, though. I’m used to being able to, like, within programming because we also sort of speak the same language, I can pick up a new programming language, and I can sit there and go through it. But for like deep learning, you’re reading academic papers, and a lot of them.
Unfortunately, the language they use in that industry is completely different than what I was used to, even coming from a math background. It took me a while just to get used to the terminology. One paper would lead to another paper, which would lead to another paper. You had to go back in time in order to sort of understand where the origins of some of these ideas came from in order to understand where they were today.
I’m still on that journey. I definitely have gotten dilusions of grandeur right now. I’m trying to build a Go AI.
CARIN: A Go AI? What does that mean? For the game Go, you mean?
CREIGHTON: For the game Go.
CARIN: Oh. Oh, okay.
CREIGHTON: Yeah. I came up with a way to use convolutional nets to sort of represent the probability of the next move.
CARIN: Ah, okay.
CREIGHTON: And I’m like, wow, I could use that as sort of like a beam search, which may make sense to some of you, but the idea would be you use the convolutional net, which just basically takes a picture of the board and figures out the patterns on the board.
CARIN: Mm-hmm.
CREIGHTON: If you feed it enough games, it’ll tell you the probability that the winning move or the next best move. It’ll tell you the list of moves in probability order, so you can then take the top four moves and do it again.
CARIN: Okay.
CREIGHTON: In essence, it’s constraining the search space because Go has billions and billions. Its search space is so huge. That’s what makes it difficult. It’s kind of like chess only way more dimensions as far as how much the search space is. This would allow you to sort of constrain it and create sort of a beam search.
CARIN: Oh, okay.
CREIGHTON: …I can do that with, like, I did calculations and stuff. I can only probably look five or six ahead on my laptop with the method that I’m doing. But if I had a thousand machines, I could really have….
CARIN: You just need Google’s resources, right?
CREIGHTON: That’s what I said. I’m like, I’m actually – later on today I’m actually talking with a company who’s specializing in sort of a super computer architecture for data centers. We’re going to be talking Clojure and stuff. I have joked with them. I said, you know what? You have all the resources that I need.
[Laughter]
CREIGHTON: We can make something magical here.
CARIN: If something comes out there and is beating Google’s DeepMind Go champion and they don’t know what it is, we know where to point, right?
CREIGHTON: Oh, that would be so cool. That may be beyond my ability. I really just want something that can beat me, which I’m pretty sure won’t be too difficult. If I then can put it on some of the Cognitect Go people, that’ll be even more fun.
CARIN: There are quite a few people at Cognitect that are very into Go. I know Fogus and David Nolen are a couple of them. I really have no idea what’s going on. I know there are stones, and they move around, but I’d be glad of an AI to help me out with any moves if I have to play.
CREIGHTON: I literally have learned the basic rules within the last three weeks, like last month. I came up with the idea of how, and I really didn’t understand Go. I started just playing a little bit and reading. I had to come up with the rules in order to sort of parse boards and do all that stuff. I am just beginning to understand. It is really complicated.
CARIN: Are there any resources that you would recommend for someone that’s interested in learning the game and learning the rules?
CREIGHTON: Actually, there was. I can’t off the top of my head, but–
CARIN: We can post them later after the show, too.
CREIGHTON: I absolutely have a list that I use. There’s one that basically is an online that goes through the very basics. It’s interesting. Go is broken down into these patterns. They’re like problems that you solve. If you see this pattern, here’s how you respond to it.
There’s this whole level system that the fact that you know how to respond to a certain pattern and you can recognize it and that really tells how good you are at Go, in a way. There are some resources that go through the first level of patterns. But while looking at that I realized, wow, you could build an AI because AIs are great at recognizing patterns.
CARIN: Yeah. Yeah.
CREIGHTON: That’s something they do well. That’s sort of a journey. It’s an ongoing thing when I get free time.
CARIN: Yeah.
CREIGHTON: This weekend I’ll be working on it if I don’t have any wrestling tournaments. It’s the first weekend in a while without wrestling tournaments.
CARIN: What other things in AI are you interested in other than the Go stuff?
CREIGHTON: Definitely some of the natural language stuff. I know that a lot of the deep learning stuff, that’s more associated with things like convolutional networks and recursive networks, which are starting to be used in some of the natural language space. But there’s a lot in the natural language space that isn’t deep learning. It’s more statistical based or, while it is neural networks, they’re much simpler, so a lot of the Stanford tools and stuff. I’m trying to–
CARIN: What Stanford tools are you talking about?
CREIGHTON: There’s a couple different ones like the Part-Of-Speech Tagger from Stanford is really nice. A lot of the stuff out of Stanford, like I’ve been going actually through the lectures for even their AI and deep learning classes because they’re online and they’re great resources.
CARIN: Mm-hmm. Are they free?
CREIGHTON: Absolutely.
CARIN: Wow. Okay.
CREIGHTON: You can just go onto YouTube and watch them, so I’ve been going through those. Now they aren’t for the faint of heart in the fact that, like, I will watch one, and then I have to go watch several other YouTube videos in order to remind myself how to do some of the math. I had to relearn some calculus and relearn things like gradient descent and logistical regression and stuff like that, stuff that I knew from school, but when you’ve been away from it a long time you don’t remember that you know that stuff.
CARIN: I’m reminded of that every time I have to help my kids with homework. I’m like how do you do long division again?
CREIGHTON: Yes. Definitely, I’ve spent a lot of time just watching other YouTube videos, but basically I use those to tell me what I should go look for and learn in order to move on to the next lecture. It’s a very slow process, but it’s been very rewarding. The Stanford’s are some of the better ones I’ve gone through. I’ve slipped in a couple different ones. The Stanford ones are really good. They have a deep learning one that I thought so far has been really good. I’ve been enjoying it.
Some of the Stanford, it’s called Stanford NLP, which is the natural language stuff. You can actually research it, but it does all kinds of things. One of the things that I really wanted to understand was how to extract knowledge. It’s one thing to say that’s a noun, that’s a verb. It’s a whole other thing to say, you know, “David runs fast,” implies that David can run.
CARIN: Mm-hmm.
CREIGHTON: And what does that mean? It’s a whole different thing to sort of extract relationships and understand that. There are tools in that that I’ve played around with recently that, while they’re not perfect, do a really good job of sort of extracting knowledge relationships. There are a couple of different tools. I’m trying to remember. They all have acronyms, so there’s NER, which is Name Entity Recognition.
CARIN: Oh, right. Mm-hmm.
CREIGHTON: It replaces pronouns.
CARIN: Mm-hmm.
CREIGHTON: That’s like the first step, and then the next one is ERD, Entity Relationship something. But it’s what runs after you replace all the pronouns.
CARIN: Okay.
CREIGHTON: But it basically returns, for a sentence, an actual tuple that says here is the subject and a relationship to another object. You can use that as the basis to sort of build things like expert systems and stuff.
CARIN: Hmm.
CREIGHTON: Then you now are starting to build up knowledge that, okay, if this implies this, this implies this. You can then use that to feed a system that can then get smarter. Maybe it can use those relationships and some other AI to basically write things like news, stuff like that, like those types of things.
CARIN: Yeah. Yeah, language is such a tricky thing, though, and especially English. I have a hard time just understanding it myself. Trying to teach a computer to understand it is–
CREIGHTON: Yes, I would absolutely agree with that. It is the one thing that surprised me the most is I have spent an enormous amount of time over the years, because of this whole grammar thing like with dyslexia, trying to figure out English. The fact that I can teach a computer to understand it better than me blows my mind. Absolutely. It’s one of those things where like when that grammar checker worked, I was just like I have done magic because to me it’s magic because I can’t do it.
CARIN: Wow.
CREIGHTON: I may even know the rules, but I can’t see the problems.
CARIN: Yeah.
CREIGHTON: And it just does it. It is one of those things that AI sometimes can look like magic. It really can. Especially in places where people struggle. There are things that we’re great at that we can see, but for things like natural language, the beauty of things like A Part-Of-Speech Tagger, I am shocked at how good things have gotten. Google’s new engine is now as good as people at actually parsing speech.
CARIN: Are you talking about for corrections or for translation to a different language? I guess both, right?
CREIGHTON: Their core thing, their parse tree that basically tells you the structure of a sentence, including all of its parts of speech, is extremely close, the closest you can get to consensus among experts.
CARIN: Wow.
CREIGHTON: If you sat down 100 English majors and had them diagram these sentences, they’ll probably be within 97%, and it’s darn close to 97%.
CARIN: Wow. Yeah, I wonder how it handles. I guess it’s smart enough that it handles it, but you’re talking about the context, like a word and I’m blanking out on an example of words, but a word could mean different things depending on the context. I wonder how it figures that out.
CREIGHTON: Yeah. Their setup, it doesn’t – while it’s a recursive neural network that they’re using and similar to my idea on the Go thing, they basically use a beam search.
CARIN: Hmm.
CREIGHTON: Which actually is how you do it on the statistical side too. You use a beam search to basically narrow down the probability stream that you’re looking at.
CARIN: Can you remind me again what exactly is a beam search?
CREIGHTON: Wow! Yeah. Okay. A beam search, all it is, let’s say that, like, a best way to look at it is if you’re driving in a car. All right. A beam search basically says instead of saying I’ve got to get from point A to point B, and I’m going to check every road. No, I know the general direction I should travel.
CARIN: Hmm. Okay.
CREIGHTON: And if I know the general direction I should travel, if I just limit my search to always be facing that general direction, I may not hit the optimal, but I will get close.
CARIN: Okay.
CREIGHTON: That’s the idea. The idea is that you’re narrowing down the search space. If you think about when you’re driving, it’s simply saying, all right, I know this is east, so I’m going to try to stay going east.
CARIN: Oh, okay.
CREIGHTON: I’m going to limit my turns and roads. I don’t want to turn west, so I won’t turn west. It just says I’m limiting the space. That’s sort of constraining the beam.
Now what makes it, why they call it a beam search is you can actually, if you have like ten choices, you may say I only want to take the top three choices. The top three are based off criteria. How close to east is it? If I want to take these three roads, how close to where – how much are they facing the object that I’m trying to go towards?
CARIN: I see.
CREIGHTON: The size of the beam is basically the size of the number of roads you’re willing to check to go down to see if they’re going in the right direction.
CARIN: I see. Okay. Huh. I know I’ve heard of it a lot, but I could never remember which one it is.
CREIGHTON: I didn’t actually know what it was. I actually implemented it in that Part-Of-Speech Tagger, and I didn’t realize that’s what it was.
CARIN: Oh, nice.
CREIGHTON: I only knew later on that that, like when I was reading other AI stuff and they were describing it, I realized that it was the exact same algorithm, and they called it a beam search. It’s fairly common in AI because a lot of our challenges in AI is trying to narrow the search space.
CARIN: Yeah.
CREIGHTON: Because if you’re trying to figure out, even – a lot of it is trying to narrow down the search space. If you think about it, logistical regression is used a lot to try to figure out what are the highest probability things and rank them.
CARIN: Mm-hmm.
CREIGHTON: The idea there is that, alright, you can actually feed that back through a thing with just the top items to actually tell you what your beam is - those type of things. There’s a lot of things that use this type of search mechanism.
CARIN: Is that like optimization? I guess that would be like – is that an optimization method?
CREIGHTON: Oh, yeah.
CARIN: Okay.
CREIGHTON: For that Part-of-Speech Tagger, it was interesting. When I first wrote it, I didn’t have the beam search in, and it basically just did a full – like if you think about, okay, this word has these three things it could be. It could be a noun. It could be a verb. It could be an adjective or whatever. The next word has these three things. You’re basically looking for the highest probability trajectory for the entire sentence. What’s the highest probability? If you look at all the words and all the types that they have, what is the highest probability of all the combinations?
CARIN: Mm-hmm.
CREIGHTON: That is the N factorial. It really quickly explodes, like really quickly. I ended up with, oh, let’s see. When I first did it, a long sentence, I was doing small sentences. I was doing a small sentence. But then I typed a long one and suddenly – I think I sat there for a minute.
CARIN: [Laughter]
CREIGHTON: I’m like, oh.
CARIN: This isn’t going to work.
CREIGHTON: This can’t be how people do this. It ain’t going to work. I started thinking about how do I actually pull this off, and I came up with the idea of a beam search with backtracking, which basically it does the beam and it starts looking. If it gets to any consistency, it increases the beam size.
CARIN: Oh, okay.
CREIGHTON: And starts looking, so the beauty of that is because English is very contextual, meaning the words around us play way bigger roles than the words farther away–
CARIN: Mm-hmm.
CREIGHTON: –that beam never gets very large–
CARIN: Oh, okay.
CREIGHTON: –before it comes to a consensus. It actually, like the one that I implemented in the Part-Of-Speech Tagger will always get to the right answer because it backtracks and increases the beam until it comes to consensus. But it literally starts out with a beam of two, and occasionally it’ll go up to five.
CARIN: Is this the same Parts-Of-Speech Tagger you have in like the Web–?
CREIGHTON: Yeah.
CARIN: Oh, okay.
CREIGHTON: Yeah, it’s integrated into – what’s it called? Hold on. I’ve got to bring it up. I don’t have my browser even open.
CARIN: Okay. Maybe we can show a link for that at the end of the show.
CREIGHTON: Yeah.
CARIN: That’ll be cool.
CREIGHTON: We can show a link to the Part-Of-Speech Tagger that I have online.
CARIN: Great. Okay. That’ll be interesting.
CREIGHTON: It’s actually go through, and the code is fairly straightforward on how to do it. I wrote it to basically give an introduction to how to do statistical learning.
CARIN: Oh, okay. Is that out on GitHub too?
CREIGHTON: Yeah.
CARIN: Cool.
CREIGHTON: So it’s well common to try to give people a gentle introduction to this.
CARIN: Nice.
[Laughter]
CARIN: Yeah, gentle is what I need there, so that would be awesome. I like math. Math is nice, but it’s been a few years since I’ve been into it deeply.
CREIGHTON: Ugh. Yeah, your background is in physics, isn’t it?
CARIN: My background is in psychics, but it’s a background that’s pretty far away at this point.
[Laughter]
CREIGHTON: So is my math background.
CARIN: Yeah. It’s 2017 now. What are you looking forward to in the next year, in tech stuff, just going on, anything that you’re kind of excited about?
CREIGHTON: Yeah, I mean definitely some of the new stuff in ClojureScript that just came out really is nice. The idea that we can now infer externs and stuff like that are going to simplify some of the things that we do. I think that the new structure that Reframe has put out, like I just implemented it at a client, and it is probably the cleanest architecture I’ve seen on the front end.
CARIN: Nice.
CREIGHTON: I’m excited to build stuff with that. That’s good. Other things, definitely still in the deep learning. I want to get a better understanding of recursive networks and stuff like that. I haven’t quite parsed through how they work. I mean I conceptually understand it, but the math is just beyond me right now. I have to sort of dig into that.
Other things, I don’t have any big open source plans other than I’d like to get my game engine to run on mobile phones and stuff like that and just see what it does. If I write a game, that’ll be awesome. But it’s definitely lower priority than some other things that I’m playing with.
Outside of that, I’ve been looking at Elixir and some other things because I still am a language nerd. There are some very cool things. I have not found anything nearly as productive as Clojure. That’s why I love Clojure.
For me, and I know it’s not for everybody. I understand that everybody’s brain works differently, and I have friends that struggled, but it literally is the closest way to the way I think. Because there’s not a lot of typing, for me programming is a lot more thought than it is typing. I tend to think a lot before I start doing something, so I end up typing really fast and getting it all out there. The beauty of the REPL and stuff like that allows me to do it really fast and to get immediate feedback as I’m going.
CARIN: Nice.
CREIGHTON: That I’m on the right track.
CARIN: Mm-hmm.
CREIGHTON: It’s definitely one of those. I enjoy Clojure. I definitely enjoy what I do. We have the beauty of being in an industry that there’s so much to learn and there’s always something new coming out. I love it. I love what I do. I cannot complain.
I get to work in my basement, which is awesome. Some people will be like, oh, I couldn’t work from home. Oh, I love it. I’m a bit of a homebody. It allows me to sort of spend more time with my kids. It definitely is my most productive area. As somebody that suffers a little bit from ADD and self-medicates with coffee, not having a lot of distractions is very, very nice.
CARIN: Yeah. Working from home definitely has its plus sides and minus sides, so I’m glad that it seems like it’s got more plus sides for you.
CREIGHTON: I don’t have a lot of negatives. My kids, like my youngest is nine, but they both – they’ll definitely leave dad alone when he’s working.
CARIN: Oh, good.
CREIGHTON: I don’t get any distractions down here. I am in the basement, which unfortunately is unfinished. I have a heater next to me because it’s cold. But that’s why I think they don’t come down here.
[Laughter]
CREIGHTON: It still has that “I’m a little scary” basement feel. There’s some advantages to that.
CARIN: Then you have a very short commute, right?
CREIGHTON: Oh, yeah. Yeah. A whole two flights of stairs. It’s awesome.
CARIN: And you do get out of the house occasionally, right, for your other activities.
CREIGHTON: Oh, yes. Obviously I do go out to Cincy FP, which is the Cincinnati Functional Programming Group, which is an awesome group. Anybody in Cincinnati or the surrounding area that’s listening to this, please come.
CARIN: Yes, you get to hear Creighton actually give some of his talks in person, which are wonderful.
CREIGHTON: Yeah, I’m actually talking in February. On Valentine’s Day, I will be speaking.
CARIN: Nice.
CREIGHTON: On Clojure architecture, architecting in Clojure. Building with Legos is what the title is going to be. I’ll be speaking there. Cincy FP, for those who don’t know, was actually started shortly after the first time I met Carin Meier.
CARIN: Yeah. I remember that.
CREIGHTON: Yes. I had just started learning Clojure, and I fell in love with it and showed up at a presentation for Clojure that Carin was doing at the Java user group. She was the first person I met in Cincinnati that wasn’t one of my close friends that even knew what Clojure was.
CARIN: Yeah, I remember that because I think I asked the question. At the beginning of the presentation I said, does anybody else have any experience with Clojure? And your hand actually went up. And I said, I want to talk to you afterwards.
CREIGHTON: Yes, and that’s actually how the Cincy Functional Programming Group started. It started because, after that, four of us got together: you, me, Ben Kyrlach, and Joe Herbers got together and decided to start it.
CARIN: Mm-hmm.
CREIGHTON: But I remember your presentation. It was the first time I had been exposed to your sort of storytelling presentation style. I was mesmerized. I was like, wow, that is so cool to do talks that way. I don’t have the storytelling ability the way you do, but wow, like that. I’ve seen a couple of your talks since then, obviously. I get to preview all of her talks before they go out because Cincy FP, she gets to speak there first, so that’s always fun. But, yeah, some of your talks have been amazing.
CARIN: Yeah. We’re lucky. We have a good group, so definitely if anyone is in the Cincinnati area, it’s the second Tuesday of the month. Feel free to stop by. We’d love to have you.
I guess we’re getting near the end of the show. As a wrapping up thing, we customarily ask our guest for any piece of advice. It could be anything at all.
CREIGHTON: Probably the advice I can give to anyone out there is never stop learning. I love what I do. I love programming. I love solving the problems. But part of that is because I get new challenges every day, and I love learning new things. To be good at this, you really have to.
You have to be learning all the time because our industry changes so fast. I would like to suggest that we’ve figured out how to do it right, but I don’t think we have. That’s the beauty of it. We’re in such a young industry that we’re still figuring out the right way to do things.
People joke around about, oh, if software engineers built bridges and all this other stuff it’d be crazy. But the reality is we’ve been building bridges a lot longer than we’ve been building software. Maybe eventually we’ll understand it enough that we can build bridges – we can build software the way we build bridges. But until then, it takes learning every day to figure out a better way to do things.
After more than 20 years of doing this, I’m still working at it. I definitely feel like I’m better than I ever have been, but I am still looking for the right way to do things every day. Every time I get to interact with new people at a client or new people from Cognitect or even just the friends at Cincy FP, every time I get to sit down, I learn so much from people. Every single person I’ve come in contact in technology has taught me an enormous amount.
I worked with a guy named Joel Holbrook. Joel, if you’re out there listening to this, awesome. His mind works so different than mine that I was amazed. He was a brilliant engineer. I kind of expected that we thought similar. You know? We’re like, well, you know, I’m an engineer. He’s an engineer. We should think similar. No, we didn’t think anything alike, and it was amazing to watch him work and his thought process taught me an enormous amount about how to approach problems differently. Because he thought so different, it was a way that it taught me that, okay, get out of my box and think about this problem this way because there are other ways to solve it. He was huge in sort of breaking problems down into their littlest, tiniest piece. I learned an enormous amount working with him. But that’s the advice I have. Never stop learning and learn from the people around you.
CARIN: That’s a great perspective and advice for definitely our industry. Well, it’s been great having you on the show. I appreciate you coming and taking time out of our day and chatting with us, and hope to have you on again because you’re into all sorts of interesting things.
CREIGHTON: Awesome. Thank you, guys, for having me.
CARIN: And thanks, everybody, for listening. This has been The Cognicast.
[Music: "Thumbs Up (for Rock N' Roll)" by Kill the Noise and Feed Me]
RUSS: You have been listening to The Cognicast. The Cognicast is a production of Cognitect, Inc. Cognitect are the makers of Datomic, and we provide consulting services around it, Clojure, and a host of other technologies to businesses ranging from the smallest startups to the Fortune 50. You can find us on the Web at cognitect.com and on Twitter at @Cognitect. You can subscribe to The Cognicast, listen to past episodes, and view cover art, show notes, and episode transcripts at our home on the Web, cognitect.com/cognicast. You can contact the show by tweeting @Cognicast or by emailing us at podcast@cognitect.com.
Our guest today was Creighton Kirkendall, and you can find Creighton on Twitter at @crkirkendall. Our host this week was Carin Meier, who is @GigaSquid on Twitter–think eight billion arms– on Twitter and GitHub.
Episode cover art is by Michael Parenteau. Audio production is by Russ Olsen and Daemian Mack. The Cognicast is produced by Kim Foster. Our theme music is Thumbs Up (for Rock N' Roll) by Kill the Noise with Feed Me. I'm Russ Olsen. Thanks for listening.