Articles

Technology The New Normal How We Work Customer Stories Testing | All Topics

Unlocking hidden value in your data

"Sustainable competitive advantage has to be won by creating the internal capacity to improve and innovate - fast and without letup." -- Spear, The High-Velocity Edge

Today, we are making available Vase, a tool we use to unleash our team’s data-driven superpowers.

The constant evolution of technology has a direct impact on business - innovate and deliver value or be left behind.  There's a lot of business value buried in your data.  The quicker and easier it is to unlock that data, the faster you get at that value and use it to do great things for your company and your customers.

At Cognitect, we live and breathe data-driven innovation. Every day we help organizations:

  • Unlock the potential of their data
  • Move into new markets quickly
  • Deliver measurable value

Vase is an example of the ways in which our teams find solutions to these challenges.  Microservices that used to take weeks or months to create only take minutes with Vase. While Vase has proven to be a valuable tool for two years as we continually developed it, it is still beta software, which will continue to evolve.

Vase: Data-driven microservices

Vase is a library for writing declarative, data-driven microservices.  A single HTTP service, complete with database integration and data validation, can be created within minutes.

We achieve this acceleration through Vase’s declarative nature- Vase does all of the mundane data-plumbing of a service, so you can focus on delivering value to your customers.  The microservices we build with Vase easily evolve and grow to meet new business demands.  Individual teams can each evolve their Vase services independently, ensuring that no team is blocked from delivering value.

A Vase Service describes three core parts: your data model, data validation, and HTTP API endpoints.  In upcoming blog posts we’ll walk through how to write each of these sections. To get you going in the meantime, we’ve got a Vase “Todo” sample as a guide and other basic documentation.

Getting started with Vase

Details for getting started with Vase can be found on the project’s GitHub page.  The Getting Started guide will take you through project creation (with the provided Leiningen/Boot template) and general development.

We’ll happily answer questions on the Pedestal mailing list, or on the #pedestal Clojurians slack.

Contact us to find out more about how Cognitect's teams of architects and developers can help your organization unlock the potential of your data. 

The New Normal: Tempo, Flow, and Maneuverability

Tempo. Most people are familiar with it in the musical sense. It’s the speed, cadence, rhythm that the music is played. It drives the music forward - and pulls it back.

But there’s more to tempo than a musical beat. In life, as author Venkatesh Rao described in his book, “Tempo,” it makes for some of the most memorable moments as it shifts faster or slower. In war, like in business, tempo - the speed at which you can transition from one task to the next - is a critical component for victory.

The New Normal: Failure Domains and Safety

Through this series, we've talked about antifragility, disposable code, high leverage, and team-scale autonomy. Earlier, we looked at the benefits of team-scale autonomy: It breaks dependencies between teams, allowing the average speed of the whole organization to increase. People on the teams will be more fulfilled and productive, too. These are nice benefits you can expect from this style, but it's not all unicorns and rainbows. There is some very real, very hard work that has to be done to get there. It should already be clear that you must challenge assumptions about architecture and development processes. But we also need to talk about critical issues of failure domains and safety.

The New Normal: Data Leverage

Like many developers, I’ve spent a lot of time thinking about objects. I started doing object-oriented programming (OOP) in 1989, and was arguably still doing it up until 2012. The concept of objects arose in GUIs and simulations, where it is natural to think about sending message to things. Over time though, the idea of messages took a back seat to the idea of locking data up in little containers where each class provides a namespace of functions to operate on it.

The New Normal: Team Scale Autonomy

You’re probably familiar with the concept of the two pizza team. This is Amazon founder and CEO Jeff Bezos’ rule that every team should be sized no bigger than you can feed with two large pizzas. At Cognitect, we take this concept one step further: the one chateaubriand team—with sharper tools you can afford to spend the extra money on better food.

The two pizza team is an important concept. The idea is that smaller teams are more self-sufficient and typically have better communication and a greater focus on getting things done because they eliminate dependencies. Every dependency is like one of the Lilliputian ropes that ties Gulliver to the beach. Each one may be simple to deal with on its own, but the collection of 1,000 tiny threads keeps you from breaking free. Eliminate dependencies and your teams move faster. Encourage autonomy and you allow innovation.

The New Normal: Everything Relies on Sharp Tools

You wouldn’t use a toothpick to wear down a mountain. You wouldn’t wear your trainers instead of your racing spikes at a track meet. So why would you use a hand saw to build an infrastructure when you need a scalpel for surgical precision?

Up to this point in the series, we’ve talked about embracing failure, experimenting more and moving faster. We've talked about the "why," now it's time to address the "how." 

The New Normal: The Art of War, Maneuverability, and Microservices

The word "antifragile" may be recent, but some of the concepts are ancient. In "Art of War", the renowned general, strategist and tactician Sun Tzu's states, “…water shapes its course according to the nature of the ground over which it flows...” In an antifragile organization, we want to explore opportunities so resources flow like water into the things that are working, and abandon those that are not. 

Just as water retains no constant shape, there are no constants in an antifragile organization and IT infrastructure. To flow like water, you must be able to shift people and teams easily, create teams and systems easily, be able to tear down systems and remove people from working on projects that aren’t working. This requires an architecture that allows you to act locally but think globally. Some organizations are pursuing microservices to this end. Complex applications are composed of small, independent processes that focus on doing a single responsibility. With microservices, developers decouple software into smaller single-function units that can be replaced individually.

The New Normal: Minimize Risk by Maximizing Change

I once worked for a startup called "Totality." Our business was outsourced web operations for companies that either didn’t want to invest or lacked the skills to build and staff their own24x7 operations center. We handled all the production management, change management, incident management—essentially the entire ITIL (IT Infrastructure Library) suite of processes.

During my time at Totality, we observed that nearly 50% of all our software outages happened within 24 hours of a software release. Since we were on the hook for uptime, but not new features, our response was obvious: Stop touching things!

The New Normal: No Silver Bullet

No Silver Bullet

In his paper "No Silver Bullet — Essence and Accidents of Software Engineering,” Fred Brooks argues that "there is no single development, in either technology or management technique, which by itself promises even one order of magnitude [tenfold] improvement within a decade in productivity, in reliability, in simplicity." Brooks, a Turing Award winner and the author who also brought us “The Mythical Man-Month: Essays on Software Engineering,” also states that "we cannot expect ever to see two-fold gains every two years" in software development, like there is in hardware development (Moore's law). In other words, there is no silver bullet.

The New Normal: From Resilient to Antifragile

Some things benefit from shocks; they thrive and grow when exposed to volatility, randomness, disorder, and stressors...

We all recognize the traditional approach to risk management: prevent problems from ever occurring. If a process fails one time, institute a review step to make sure it never fails that way again. The next level is resilient systems that can cope with change and survive failures. I propose that the new normal is the exact opposite of preventing problems: survive problems, and if you’re not getting enough problems from the outside, make problems for yourself!  This new approach is still counterintuitive in most organizations.

The New Normal: Failure is a Good Thing

It seems that everyone is talking about microservices as the road to salvation. Why? Why now? The usual explanation is just that it's an architecture style that encourages flexibility and makes a company more competitive. However, like agile development, patterns, and object-oriented development before it, the microservices architecture will not deliver its promises to everyone. If you understand what makes it work, you will know when and how to apply it successfully. Let us look deeper to see what forces really underlie this evolution.