As a part of ProgrammableWeb's ongoing series of on-demand re-broadcasts of presentations that are given at the monthly Washington, DC-Area API meetup (anyone can attend), this article offers a recording and full transcript of the APIs 101 presentation that I gave on Dec 5, 2019. It's the first of a long-running series of 101 classes that I'll be giving at the meetup. In the presentation, I discuss how the API contract essentially decouples the API consumer from the API provider and the sort of flexibility that comes with that Decoupling.
The DC-Area API Meetup almost always takes place on the first Tuesday of every month. The attendees consist of API enthusiasts and practitioners from all around the federal government as well as businesses and organizations that are local to the DC Metro area. There is no charge to attend and attendees get free pizza and beer, compliments of the sponsors. The meetup is always looking for great speakers and sustaining sponsors. If you're interested in either opportunity, please contact David Berlind at David.Berlind@programmableweb.com. If you're interested in attending, just visit the meetup page and RSVP one of the upcoming meetups. It's that simple.
Here's the video of my presentation:
What is an API and Why does the API Contract Matter So Much?
Editor's Note: This and other original video content (interviews, demos, etc.) from ProgrammableWeb can also be found on ProgrammableWeb's YouTube Channel.
Editor's note: ProgrammableWeb has started a podcast called ProgrammableWeb's Developers Rock Podcast. To subscribe to the podcast with an iPhone, go to ProgrammableWeb's iTunes channel. To subscribe via Google Play Music, go to our Google Play Music channel. Or point your podcatcher to our SoundCloud RSS feed or tune into our station on SoundCloud.
Transcript: What is an API and Why does the API Contract Matter So Much?
David Berlind: So, as Gray said, one of the things that we've got going on here is we're going is I'm going to be delivering an ongoing series of talks, which is more like a kind of a college course on APIs. I think 101 maybe is—probably a bit of a misnomer—because it does cover some fairly interesting nuances of API's. But the way this will go is I'm going to give a talk here. And I have the whole series planned out and most of the content developed. But, I'm only going to give a part of it here, and then I'll give the continuation of it, and the ongoing continuation of it at each of the subsequent meetups. So, you're seeing the first part, and then in January you'll see what comes next, and so on and so forth.
These talks, since I know some of you, you're experienced practitioners with API, so some of this is obviously going to be very rudimentary for you. But, I think to Gray's point, one of the challenges that anybody has in an organization, even if you really know your API stuff, is explaining it to the other people in the organization so they understand it. And, over the last nearly decade of working in the area of APIs, one of the things I've come to learn working with different organizations is that it doesn't really do the organization a whole lot of good if it's the IT department or somebody in the IT Function driving the API strategy. The other side of the organization, the business people, really need to understand what the power of APIs are and how they can be so transformative to a company to make a company into more of a Platform if you want to call it that. And, so maybe while some of this will be review or rudimentary for you, it may spark some ideas on how to talk about APIs with other people so they begin to understand the benefits in terms that they understand.
So, what I'm going to go through across this entire series will be "What exactly is an API?", " How do they work?", "What are the key benefits of Web API architecture?" By the way, this is all tweetable. You can tweet all this stuff and tell your friends [that] this is a good place to come. If you don't know about APIs, you don't have to be an expert in them. Great place to come to learn about them. "What are the different styles of APIs and when to use them?" This is something that is near and dear to my heart because I'm the editor in chief of ProgrammableWeb, which a lot of people come to. They know it as the "Journal of the API Economy", but we also run the world's largest independent directory of public APIs. And part of the way we categorize them is based on their styles, and there are different styles of APIs.
That's should be "Why are the APIs the engines for Digital Transformation of organizations and business reinvention?" This is where you get into that area of why is it really relevant for business people to understand how APIs can benefit the business and change the business. Because every business needs to go through these changes if they want to stay competitive in the new world. "What are the different business models?" And, actually, I can't even answer this question right now because we're in the process of finishing this report. But, the definitive reference for the business models for APIs is a presentation that was given the founder of ProgrammableWeb, a gentleman by the name of John Musser. And the title of the presentation, you can look it up, you can Google it, it's "20 API Business Models in 20 Minutes". And people look at that as the sort of gold standard reference of how to understand the different business models. Well, it turns out that it was published back in 2013. So, a lot of time has passed, especially in internet years, and things have changed. And so, we're actually putting out a whole new comprehensive guide to all the different business models, and what makes them different from one another.
"How and why you should be productizing APIs?", and "What are the different API security concerns, and how to address them?" And, "What is API first design, and why is it important?"This is what I've gotten on the list so far, but it's going to expand. Because as we go on, people are going to have questions and say, "Hey, you gave a presentation on X," and so I will add it to the curriculum.
So first, I just want to talk about, this is one, Gartner actually adopted this phraseology from my presentation, which is that if you think of the cloud that's in between the smartphone, and what is our icon for the server here, right, in that role, all of us are used to using these, right? You all have one of these. And this has a User Interface on it, and it's intended for a human, right? So you all know how to use it, you know how to tap it, you know how to scroll it, you know how to operate the applications that are in here. And it's rather intuitive, or in some cases not, but you get to figure it out. And that is a user interface. It's designed for human consumption. And, in that role when you're actually using your smartphone, it is communicating at some point with some other server that's across the internet on the other side of the cloud. So, the only difference between what you're doing with your smartphone and what an API is, and by the way they're API is involved when you do those things, is that the machine is also a user. But, the user interface that it needs to talk to the server on the other side just has to be designed for a machine to consume it instead of a human to consume it.
And so, what I always like to say is well look, these are both users. They're just different types of users. One is a human user, the other one is a machine user. And an API is really just the user interface that's designed so that machines can consume it. And this allows two machines to really talk to each other, or two pieces of software to talk to each other. A lot of times you hear other people describe it that way. It allows two pieces or more of software to talk to each other. So, there's something called a contract that's involved in this conversation that takes place between two machines or two pieces of software.
But, first I want to talk about some basic terminology that I think will be useful to you. We have this world in the API economy that consists of consumers and providers. And some people get consumers confused with, oh, consumer, like, consumer in the retail world, the consumer. No, that's not the case. In a software application, like the ones that you use in your smartphone, when they're talking to another piece of software, and they're going through an API, that is the consumer. So the actual application that is talking to another machine is the consumer of the API. And the machine that's providing that API is the provider.
The company or the entity that runs that machine on the other side that is providing that API that the consumer is talking to is often referred to as the API provider. And so, these consumer, provider, also sometimes you think of the consumer as the "developing consumer." You can think of developers who build applications and use the APIs as the initial consumer of the API because they're writing or coding the application that consumes the API. So developing consumer, consumer, provider, API provider. These are important terms that we try to stay consistent with when we're talking about APIs.
Now, a real basic part of this, and some of you are very familiar with this of course, is that oftentimes an application outsources some function to an API. And we often call that "calling" the API. It's like picking up the phone and making a phone call. Only this time the software is calling the API, and the API is going to respond with whatever it's designed to respond with. If you're calling the Google Maps API, it responds with an actual map that gets painted on your screen. So if you've used to using Uber or Lyft, you see the maps getting presented right in the application. They're calling the API of some mapping API provider. You might be calling API for something like a patient record. I think we had somebody here is with Medicare and Medicaid, so you know all about that kind of stuff, right? Like I said, a pin on a map that that functionality sort of provided by Google Maps. Or maybe, a financial transaction, a banking transaction of some sort. There are a lot of different API providers that offer API based financial transactions. PayPal, for example, does that, Dwolla does that. Stripe does that. A lot of banks do that.
And, to make this work, there's this "contract" that exists between the API consumer and the API provider. And just like a legal contract, there's a mutual understanding between the consumer and the provider for this to work. The provider is on the hook to make sure that they live up to the contract and do exactly what they say the contract says they should do. And there is some burden on the developer to do the same thing, to make sure that they call the API correctly. And all of those things are baked into what we sometimes call a "technical contract." This is one of the more confusing areas when you're explaining it to maybe a business person, but this contract turns out to be one of the most important features of an API in terms of the benefits it has to offer to organizations. So it's like a legal contract, it represents an understanding by all parties involved. And it also represents some agreed-upon standards that the two sides, the consumer and the provider, are going to use.
And this includes even de jure standards like official standards that are out there. So for example, the standard for XML is a worldwide standard, that's well understood by everybody. And your API, the consumer and the provider of the API, when they're agreeing to a technical contract, are also agreeing, hey, the data that we share back and forth between each other is going to be transferred in the XML format, which is a standard.
So I have some analogies to help you understand the contract and help you explain it to other people. We have a plugin the wall here. We have the construction guy, the LEGO construction guy, and then my favorite of all is the intermodal shipping Container. So, here's an example contract. And one of the key things about this contract is what we call this idea of "decoupling." So the consumer is fully decoupled, this is an important concept, decoupled from the provider. And the electrical socket, each of these analogies kind of gets that point across, and the electrical socket's really good example of that. Now the electrical socket has a very specific contract in it, right? There is a very, very important arrangement of the different holes that are in the socket. You have a sort of an elongated hole which is often referred to as the "neutral." You have the shorter one which is sometimes called the "hot", and then you have that little semi-ovular one which is the "ground." And when something plugs into it and fits into that socket, and you know everybody has seen a plug that only can fit in one way, they know they're going to get 120 volts of alternating current. They'll get somewhere between 15, 20 amps through a socket like this, or up to that. It'll be, like I said, alternating current. And here's a little bit of the data on the socket configuration. And if you tore the wire apart that you're plugging in, you very often find these colors in that wire because they're part of an agreed-upon standard of what color wires you use for this.
And so is a really good example. If you think about things that you plug into an electrical socket, let's take a hairdryer, for example, that hairdryer is designed to know that when it plugs into the socket, it's going to get all of this. It doesn't care how that is delivered to the socket on the other side by the provider. It could be wind, it could be solar, it could be nuclear, doesn't matter. As long as the hairdryer gets 120 volts through a socket rated for 15 to 20 amps and alternating current, the hairdryer is happy, so this makes it possible for the designer of the hairdryer to design just that specification, and not worry about how that's going to get to the socket. And this is that issue of decoupling. The hairdryer is completely decoupled from the utility that provides the electricity. It doesn't know or care how the utility delivers this contract to the wall socket. And by the way, the utility doesn't really care about the hairdryer. They're fully decoupled from each other. They're fully insulated from one another. Right?
So let's take another example here, which is LEGO, which is probably the most common used analogy for APIs because APIs turned out to be like building blocks. And, as it turns out, there's a very strict technical contract when it comes to LEGO in terms of the size of the nubs on the little blocks. And this kind of shows you a little bit about what that specification's about, but that's a contract. And anybody who has worked with LEGO before knows that if you buy one of the fake LEGO makers, and you try to snap it on, it doesn't quite fit so perfectly as a real LEGO block will fit. Right? And that's because they don't quite have the contract down. And it's probably because there's some intellectual property thing that prevents them from copying the contract. But, point being, is that there's a contract there. And you have essentially a provider of a nub and a consumer of the nub in a way that sort of decouples and separates the concerns of the two blocks from one another. All they care about is that the provider and the consumer conform to the contract.
Now if you have kids, like I do, I have four kids, this is the new thing. Oh, not really new anymore. But when you go, at Christmas time, you go to the store and you want to buy a gift, and you see this on the shelf. And you realize that LEGO has kind of gotten away from this idea of a contract. Because one of the big advantages of decoupling is, like I said, you don't really care. Everything kind of works together no matter what. And suddenly, you get something like this, and there's a part like that that comes with it. And you're like, Jesus, you know what? This thing only works on that airplane. It's totally coupled to this design. And you can't really use this anywhere else. The contract is actually in the nubs there. But, good luck making this really work with your other LEGO pieces. So in some way, it's still decoupled because the nubs are conforming to the contract, but it's not in the true spirit of what LEGO once was a long time ago. It only really fits on that airplane. Now maybe some creative kids will come up with some other way to make this really fit into something else. And that's because the contract is still somewhat showing up there in the nubs.
Let's talk about intermodal modal shipping containers. I never knew this, and then I suddenly stumbled across, I said, wow, that's a perfect analogy for... and it's an amazing analogy for APIs, and an amazing bit of history. So this guy, Malcolm McLean, he's considered the father of modern-day shipping logistics. And, if you know something about how shipping used to work before they had shipping containers, this is the way it worked. The truck showed up at the docks, and there were a bunch of bunch of longshoremen on the docks, and they literally hand loaded the merchandise off the truck into some kind of box that was on the boat, and then the boat went wherever it went. And the exact same process worked in reverse on the other side. Terribly inefficient, and human-oriented process. Can you imagine people dropping stuff, breaking stuff, this would never work in today when we're shipping electronics and cars, and things like that all over the world?
So, Malcolm McLean came up with this better idea. He said, "what if we make it so that, let's say you're a manufacturer of goods, you load a container with some stuff. And then we bring that whole container to the ship. And by the way, it gets there on the back of a tractor-trailer truck, which goes to a train station, and puts it on a rail car, and the rail car brings it to the docks. And then it goes on to the boat. It lifted by a crane onto the boat. And there's no handling of the merchandise by longshoremen. It's just a container." So, the container leaves the manufacturer, eventually finds its way in a boat, and the boat goes to where it's going. And the container is taken off the boat and sent to its final destination. And then it's up to whoever is at that destination to unload the container. So now the burden of loading and unloading is really the very beginning and end of the process.
Well, to make this possible, they had to come up with this thing called the twist lock, right? And the twist lock is this device, this casting that is on all four corners of the container. And it allows the containers to be locked to one another, but it also allows the containers to be locked down to the tractor-trailer trucks, and to the rail car, and then the ship itself, and then to each other. Right?
Here you really have another contract in place. This is a standard, a worldwide global standard, where you have the provider of the contract and the provider of the interface and the consumer. The provider is the container. You could look at it either way, but I think the consumer is the truck because it's taking the container and moving it around. So next time you're on the highway and you see one of these trucks drive around, you see one of those containers that says Maersk on it or something like that, now you're looking at it completely differently. You're going to be looking for the twist lock, right? Because you're going to think, wait a minute, that thing right there, that's an example of an API because there's a very strict contract that everybody is sticking to in order to make this work.
And, quite frankly, the container, just like with the hairdryer, it doesn't care what it's being connected to. It doesn't care. It's decoupled. It's essentially decoupled. It could be the tractor-trailer, it could be another container, it could be a rail car, or it could be the ship itself. And so here's an example. Here's the contract of the twist lock. I don't have to go into the details. I'm not going to test you on this.
And then, the question from me to you is, what are some of the other examples of this? Because in the world around you, we are literally surrounded by examples of these standards that end up decoupling the provider from the consumer. Does anybody have any ideas?
Audience Member: Currency.
David: How so?
Audience Member: I give a bartender five bucks, he gives me a beer.
David: And, that's a contract. That's actually, there's a sort of a legally binding contract right there, right? Okay.
Audience Member: I give him seven bucks, he gives me one.
David: That's right.
Audience Member: I can go on and on.
David: I think we're going to go through the whole world of spirits and liquor before we get... yeah, okay. Any others? Come on.
Audience Member 2: Most networks. So you could say road network. It's an abstraction that allows all kinds of various things from small contact scooters to huge ATV vehicles. It doesn't care. But it follows strict standards that subsequently are then followed by manufacturers.
David: The width of the lanes, for example, right?
Audience Member 3: Width of the lanes, traffic goes on one side one way,...
David: When I've given this presentation before, people have brought up the wheel and tire. The tire's mounted to the wheel, but there's a contract that lives between the two of them. In a way that you could get tires from any number of manufacturers and put it on a wheel, right, on the same wheel.
Audience Member 4: Railroads.
Audience Member 5: Railroad tracks.
David: Right, and there's all kinds of legendary folklore about how we ended up with that. Right? You always see that somewhere. You come across the internet every two years, it makes the rounds, goes viral, how did we end up with the width of the tracks. I forget how that story goes, but if you pay close enough attention, it'll come your way eventually.
Audience Member 6: It's been debunked, though.
Audience Member 6: It's been debunked.
David: Oh, okay. You can't believe everything. It's all fake news. There you go.
Audience Member 7: Too soon, right?
David: That's right. So point being, this decoupled nature creates all kinds of great opportunities that lead to digital transformation and reinvention of a business. And we'll get to those in the future versions of this course. One thing I want to leave you with though is that APIs are not new. APIs have been around for decades actually. They just have been coming in many different styles and forms.
But just to give you an example of... I looked at the name of this meetup. I was like, wait a minute, what's the name of the meetup? And it's the DC Web API Meetup, right? And that's what it says on meetup.com. And I thought that's really interesting that the word web's in there. It's actually not the DC API Meetup, it's DC Web API Meetup. And there was something that happened, this is important in the arc of API history, where the web got involved, which introduced even better decoupling, and it was essential to how we got to where we are today. And I want to talk a little bit about that.
Now again, APIs have been around for decades, but for the lion's share of that time, they were predominantly what we call RPC APIs. And RPC stands for Remote Procedure Call. Again, we have two machines here. They're talking over the internet, or over a network in many cases, not even the internet. An RPC is in the middle. And essentially, what would happen there is one machine had to know specifically what procedure on the other machine to call. And maybe the name of that procedure was "retrieved customer info."
Now, this RPC kind of architecture has happened in a number of different ways over the many years that we have APIs in existence. But essentially that is very tight coupling. The consumer has to know something incredibly specific about the provider and the code that is written on the consumer is very tied to the procedures that live on the provider side. And it was the web API that changed everything. And so let me show you how that is.
Today, when we hear about web APIs, the communication that's taking place between the provider and the consumer is over the protocol of the worldwide web, which is HTTP and you see that in your web browser, HTTP something, something, something. Well, that stands for hypertext transfer protocol. That is interchangeable with the term the web. The two things mean exactly the same thing. There's an official protocol there. But, what was special about that protocol is that the protocol has built into it its own set of commands. So that instead of retrieving a customer, having to know the name of the procedure that's on the other side, all you have to know are the list of commands that are a part of the web's protocol.
So in this case, instead of saying retrieve customer info, you might just issue a command that says get. Get is one of the verbs, they call them verbs, of the HTTP protocol. Now if the consumer issues a get request to the provider, it's up to the provider to figure out what to do with it.
But, this completely decouples the consumer from having to know the specifics, the names of the procedures on the provider side. And this was something very special about the web that changed the world of APIs in a big way. This really kind of was what broke things open when it came to putting things like Google Maps as an API based service on the web. Okay? And Google Maps is essentially the second API that we made a record of. Or no, the first API we made a record of in the ProgrammableWeb database. The second one was Flickr, which was acquired by Yahoo. Right?
But, both of them worked across the web, and both of them used the web's protocol to issue these commands. And as soon as developers saw that, the flood gates broke open. That was really easy for them to understand. It made life a whole lot easier for them to actually start working with APIs because there's only a handful of these verbs. There's get, put, patch, et cetera. Right? And you can look that up. If RPC APIs that came before were sort of decoupling things, this was really decoupling things.
And so, this is a little bit of a hint of where that leads us for the next presentation. But, imagine if you will, that you have all different kinds of consumers. On the top here you have machines, you have web browsers, you have iPhones and Android phones, you have notebook computers all talking over the internet to a variety of things, servers or providers on the back end. Technologies that provide the API on the back end. It could be .NET, could be Linux, could be something, an IBM mainframe. The contract is that agreement.
And I'll leave you with one thing, and we'll talk about it more next time, is just imagine, let's say, you have an IBM mainframe that costs millions of dollars to run and maintain and to license the software for it from IBM. And what have you could just replace it with a Linux server in Amazon's cloud? Well, the contract is what allows you to do that. It's just a substitution that you would make and say, okay, instead of providing that electricity to the plugin your wall with coal, we're going to provide it with solar. Something that's way more cost-efficient, way more friendly to the earth, and so on.
Well, this gives organizations the power to make those substitutions. Maybe that IBM mainframe is overkill, and is not worth it anymore. And maybe you could provide that same contract to the consumers with something that's far more cost-efficient like a Linux server running in the Amazon cloud. That is real power. That could transform a company, that can transform the bottom line in ways that are incredibly significant to the company and make it more profitable, give it more control over its future, and make it more competitive in the marketplace that it's working in, because now it has more capital to work with instead of the capital that it's just wasting on a big mainframe. Not that mainframes are a big waste of money. But again, the contract is what makes this possible. Because the back end and the front end are completely decoupled from one another.
In a future version of the presentation, we'll talk a little bit about more about what it means to move from an IBM mainframe to a Linux server. That's a conversation that we call " Legacy Modernization." So thank you, and I will continue this the next time.