~ 4 min

Checking on your twitter friends

May 29, 2017 #twitter #automation

Let's be honest, I’m lazy and you're probably lazy too… I might even be the laziest person on earth but I’m still waiting for a confirmation on that one from the Guinness (the book, not the beer…). I specially hate doing repetitive tasks by hand. This might be one of the main reasons why I became a developer in the first place… That and to build cool stuff. Anyway, I'm sure the world is full with lazy people, I can't be the only one with this “problem” and this post is for them, for everyone who’d like a little bit of automation in their lives.

Automating twitter

We’ll go step by step to build a script to fetch the list of your followers (the ones who follows you) and the list of friends (the ones you follow) on twitter. To follow along, you should have a basic understanding of the terminal and python. If you get stuck, send me a tweet and I'll be happy to help you out.

Let’s get ready

To connect to the twitter API, you’ll need proper credentials and fortunately for us, they are very easy to get. To do it, go to http://apps.twitter.com/app/new, fill everything and create your new app. After the refresh, go to the Key and Access tokens tab, click Create my access token and you should be able to collect everything needed from the screen:

Consumer Key
Consumer Secret
Access Token
Access Token Secret

We now have everything, for the non-code part at least. So, let’s start the coding. To get started, create a new file and import the two libraries we’ll need:

tweepy - our script is based on tweepy. This library alone will let you make requests to the twitter API. To install it, run pip install tweepy on your terminal and you should be ready to go. If you installed python properly, you should have pip too.
time - the twitter API has limits, so you can't make thousands of requests at once. Since we plan to fetch a good amount of data (depending on your total amount of friends/followers), we'll use time to add some pauses between requests. This library is "free", meaning, it comes installed with python by default.

Since you are getting everything ready, grab your credentials and add them to the right spot on the code bellow and don’t forget to change the screen name too, or you’ll be analyzing my followers/friends.

import time
import tweepy

consumerKey = '__________'
consumerSecret = '__________'
accessToken = '__________'
accessTokenSecret = '__________'
screenName = 'fullybearded_'

Next, with your keys, we’ll initialise tweepy to be able to interact with twitter.

authTweepy = tweepy.OAuthHandler(consumerKey, consumerSecret)
authTweepy.set_access_token(accessToken, accessTokenSecret)
api = tweepy.API(authTweepy)

Fetching our followers

Now that everything is set up, let’s fetch our followers. To prevent mass extraction of data, the Twitter API is paginated just like its web interface. This means that instead of fetching a long list of followers/friends, we’ll need to fetch multiple pages of users and put everything back together into a big list. In practice, we’ll use a cursor (this comes with tweepy) to fetch all the pages of followers, and for every follower we'll keep their screen name.

Note: When making calls to twitter, we need to pace ourselves and limit the number of requests or we’ll end up hitting the limit and be temporarily locked out. To be safe, wait a bit before fetching the next page of results. To do it, use time.sleep(60) to pause for 60seconds.

followers = []
for page in tweepy.Cursor(api.followers, screen_name=screenName, count=200).pages():
    print '> Fetching api.followers'
    for user in page:
        followers.append(user.screen_name)
    time.sleep(60)

print 'followers count', len(followers)

Fetching our friends

After getting a list of followers, we’ll now fetch a list of friends. The process is pretty much the same, the only thing that changes is the function used to fetch the data. We use api.friends instead of api.followers.

friends = []
for page in tweepy.Cursor(api.friends, screen_name=screenName, count=200).pages():
    print '> Fetching api.friends'
    for user in page:
        friends.append(user.screen_name)
    time.sleep(60)

print 'friends count', len(friends)

Crossing the finish line

Now that we fetched everything, we’ll use both lists to extract valuable information: by crossing the lists we can check who’s present in one list only. To do it, we need to convert our previous lists to sets. This allows us to subtract two lists and get the elements not present in both lists. In practice, this means we’ll see who follows you but you don’t follow back and who you follow but doesn’t follow back.

setFollowers = set(followers)
setFriends = set(friends)

print 'They follow you but you don\'t follow back:'
print setFollowers - setFriends
print
print 'You follow but they don\'t follow back:'
print setFriends - setFollowers

Final code

By now, you should have this final version of the code:

import time
import tweepy

consumerKey = '__________'
consumerSecret = '__________'
accessToken = '__________'
accessTokenSecret = '__________'
screenName = 'fullybearded_'
authTweepy = tweepy.OAuthHandler(consumerKey, consumerSecret)
authTweepy.set_access_token(accessToken, accessTokenSecret)
api = tweepy.API(authTweepy)
followers = []
for page in tweepy.Cursor(api.followers, screen_name=screenName, count=200).pages():
    print '> Fetching api.followers'
    for user in page:
        followers.append(user.screen_name)
    time.sleep(60)

print 'followers count', len(followers)
friends = []
for page in tweepy.Cursor(api.friends, screen_name=screenName, count=2000).pages():
    print '> Fetching api.friends'
    for user in page:
        friends.append(user.screen_name)
    time.sleep(60)

print 'friends count', len(friends)
setFollowers = set(followers)
setFriends = set(friends)

print 'They follow you but you don\'t follow back:'
print setFollowers - setFriends
print
print 'You follow but they don\'t follow back:'
print setFriends - setFollowers

A more efficient lazy human

I hope you managed to get your script running and fetching your followers/friends without lifting a finger (isn’t that awesome?!). With that info, now you know who follows you but you don’t follow back and the ones you follow but don’t follow back. I believe this information could be put to use in many ways, helping you improve your account(s).

I hope you liked this post and if it was useful, then my mission is complete, and I was able to help you to be a “more efficient lazy human”… And remember, if by chance you get stuck, just let me know on twitter and I’ll try to help.

PS: Quick question: What do you want to learn next? What do you want to see automated? Send me a message on twitter!