Twitter API

Our Goal

This is the first tweet currently available on Twitter, authored by Twitter co-founder Jack Dorsey:

Twitter's API is used by thousands of applications today, and it was one of the earliest popular services to provide an API for use by other applications. Let's see what is required to fetch the above tweet using this API.

Reading Documentation

The first step of working with any API is collecting a few pieces of information:

• What protocol, host and path (basically, what URL) will provide access to the appropriate resource?
• What parameters do I need to include in the request?
• Is authentication required?

In order to answer these questions, we'll start by browsing the official Twitter API documentation. The left sidebar lists a few introductory sections and then a list of paths, one for each operation that can be performed. Clicking each of these paths will load a page containing more information about that resource, including the location of the resource and any requirements for interacting with it.

Partway down the left column is GET statuses/show/:id. Clicking this will display documentation for it:

Putting everything together, we now know the following about the request we need to make:

• The complete URL is https://api.twitter.com/1.1/statuses/show.json
• The parameter to send is id. (The id of the tweet we are interested in is 20.)
• We will need to make an authenticated request to access this resource.

Twitter requires all API access be done over HTTPS to improve the security of its system. This is why the scheme begins with https and not http.

Gaining Access

Now that we've collected all the information needed to fetch the tweet, let's try fetching it using Postman. We're using it instead of HTTPie because the Twitter API requires authentication using the OAuth protocol, and Postman supports sending this type of authentication.

Even though we know we need to include authentication as a part of the request, let's see what happens if we leave it out.

A 400 Bad Request status tells us the server didn't like something about our request. Looking into the JSON body gives us further clues: errors[0].message is Bad Authentication data, and looking up error code 215 in Twitter's response code documentation tells us the same thing. This should come as no surprise, since we already knew this resource required authentication.

Our next step, then, is to work through authenticating with Twitter. Starting in 2010, all requests made to the Twitter API are required to use OAuth. This went a long way to improve the security of the Twitter ecosystem, but it did make it much harder to interact with Twitter using a command-line application such as HTTPie or cURL. Whereas before this change a simple username and password could be used to make authenticated requests (just like on the web store), all interactions with the Twitter API must use a protocol known as OAuth.

What is OAuth?

OAuth is a complicated system that provides a way for users to grant access to third party applications without revealing their credentials. The basic flow begins with a user clicking a button that says something along the lines of Connect your Service Account. The user is then redirected to the service's site where they approve the originating application's access and is then redirected back to that site. There are also a few interactions happening directly between the servers that are not exposed to the user. The end result of this is an access token and an access token secret, which, when combined with an application key and application secret belonging to the requesting application, provide enough information to build a request using a somewhat involved list of steps we won't get into here.

You can think of the application key and application secret as the application's username and password, and the access token and access token secret as the username and password belonging to an individual user. These values are used for different purposes, but ultimately you need all four values to make requests.

Ultimately, OAuth provides a decent way for users to authorize access to their account without giving out their credentials, and it can be a fairly smooth experience for end users using a desktop web browser. Implementing all the functionality to make it work can be a lot of work for a developer, although there are libraries available for many programming languages that can make this process much more manageable.

Twitter has also provided another way to create OAuth tokens for use in developing new applications, allowing us to skip most of those steps by providing us an access token and access token secret for development purposes. We will be using these values throughout this chapter in order to focus on working with the Twitter API and not the particulars of OAuth.

Setting up a Twitter Application

Since we'll be posting to the Twitter account used in these examples, it is probably a good idea to create an account just for development work involving Twitter. This can be done via the signup form on the Twitter website. It's OK to skip the five step wizard once you have created your account.

In order to register an app, your new Twitter account needs to have a confirmed email address. Go to your email, find the email from Twitter, and click the Confirm Now button.

The next step is to register a Twitter application, which can be done via the Application Management page on Twitter once you are logged in. Enter whatever information you want in the application form- the values don't matter for our purposes.

Once that is complete, go to the page for the new application and change the permissions to be Read and Write.

Next we need to generate an access token for the current user, which will be you. This means you will be accessing Twitter through the API as if you were that user. This allows applications to perform actions on behalf of users, and have those actions properly handled by Twitter, taking into account any permissions on their end.

To access the tokens, go to the Keys and Access Tokens tab on the application page. Scroll down and click the button to generate user tokens.

Back in Postman, click the Oauth 1.0 tab at the top. Copy in the consumer and user tokens and secrets from the application tokens page.

Having set up these values, it is a good idea to make sure that requests made to Twitter authenticate correctly. There is a resource that can be accessed using a GET request that returns data for the user currently authenticated through the API. It will return an error status code if authentication failed.

The response has a HTTP status code of 200 OK, so we know the request was authenticated successfully. OAuth uses the same Authorization header that HTTP Basic authentication does, only it requires a lot more values:

Authorization: OAuth oauth_nonce="172914044896444049971412201185", oauth_timestamp="1412201185", oauth_version="1.0", oauth_signature_method="HMAC-SHA1", oauth_consumer_key="BSMOqjdmdhjQi7Tl5VrkEIsVb", oauth_token="14615072-vE5mKBKxTOmiX75OwRKFXIz3VdWMQqhFEoAE5qcTM", oauth_signature="M93PU0cxmUqoSp0KGQWQuiV2Rqw%3D"

The specifics of the purpose of each value in an OAuth Authorization header is beyond the scope of this book, but it is good to know what to look for when troubleshooting authentication problems. OAuth itself is a complicated system with several incompatible versions and many differing implementations. A good description of how it works can be found in The OAuth Bible.

Postman used the tokens and secrets we entered earlier and computed the values in this header automatically. When developing an application, it is common to use a library that will do something similar to make working with Oauth more straight forward.

Fetching a Tweet

Now that authentication is sorted out, let's try to load the data for the first tweet again.

The response JSON contains many more keys than the responses we were receiving from the web store. We can see that the value for text is "just setting up my twttr" and the value for user.screen_name is jack, which is what we'd expect based on the display of the same tweet above.

While some of the keys in the JSON have fairly self-explanatory names, it is common for some to have cryptic names. Even the fact that the API refers to tweets as statuses or status updates can be confusing. Good API documentation will include information on what kind of resources the API exposes and what the meaning of each value returned in a JSON body is. Twitter has a page in its documentation about tweets that covers what each key and value in a tweet response represent. Take a look through the JSON in the previous example, and try to find the explanation for a few keys on that page.

Note that the value for user is an embedded representation of a user, which has its own page in the twitter docs. Embedding related resources in this way is fairly common.

Let's take a look at what Twitter returns when a request is made for a tweet that doesn't exist. Changing the value of id to 19 (which doesn't exist) will return the desired error:

404 Not Found makes sense since we know the resource we're requesting (a tweet with an ID of 19) doesn't exist.

Posting a Tweet

Now that we've fetched tweets from Twitter, the next step is to try to post a new tweet of our own. Just as we did when fetching a single tweet, we will consult the Twitter documentation to see what URL and parameters we need to include as part of a request.

From the documentation, the information that is needed to create a tweet is:

• A URL of https://api.twitter.com/1.1/statuses/update.json
• A status parameter, which is the text of the tweet.
• The HTTP method POST since this is creating a new resource.

Update the URL and HTTP method in Postman. It is important to set the POST to x-www-form-urlencoded. This causes the parameters to be included in the request body and encoded as if they were query parameters. Then set the parameter status and give it any value you'd like.

Twitter API and REST

The Twitter API is successful in that it is used by thousands of applications and handles an extraordinary amount of traffic. From a RESTful-design standpoint, however, there are some changes that could be made to keep the API closer to the conventions of REST.

The Twitter API might be considered less RESTful than it could be for a few reasons:

• It only uses GET and POST instead of the full range of HTTP methods. This makes the API less expressive.
• As a result, resource paths include actions that are being performed on them.

There are a few examples in the following table of resources as they are exposed by the current Twitter API and how they could be exposed in an API that was created today. Every segment of resource paths that follow the REST conventions should represent either a collection or an element. In the table, note how the routes include show, update, and destroy, and how these segments could be removed by leveraging the full range of HTTP methods:

Objective	HTTP Method	Path	Improved HTTP Method	Improved Path
Fetch a single status	GET	/statuses/show/:id	GET	/statuses/:id
Create a status	POST	/statuses/update	POST	/statuses
Delete a status	POST	/statuses/destroy/:id	DELETE	/statuses/:id

So why doesn't Twitter make these changes? To start with, modifying public APIs has to be done very thoughtfully, and usually removing resources or parameters will require releasing a new version of the API. This allows consumers to opt-in to the API changes and goes a long way to preventing breaking changes from disrupting existing API clients.

In addition, while these adjustments might make the API a bit more conventional, it would otherwise not change the functionality of the API. It's hard to make the case for potentially disruptive changes that will involve a lot of work for engineers (both inside and outside of Twitter) when there isn't a functional benefit resulting from the efforts.

Most (if not all) public APIs deviate from the REST conventions in a few ways, often for historic reasons. This information should be included in the API documentation for most services, so when in doubt, check the docs!