Week 2 - Log my Twit using Twitter API and Python

This week we’ve started an experiment using Twitter’s Streaming API. 

• See Twitter API for further information: https://dev.twitter.com/streaming/overview
• App account creation: https://apps.twitter.com/

The Streaming APIs give developers low latency access to Twitter’s global stream of Tweet data. We needed to use Streaming API to access Twitter’s global stream of Tweet data for multi searches. Our Twit collector will tap into the Public Stream and will be tapping on behalf of a single user. This will just be a single collector agent. The collector will connect, get tweets, parse data and store for future processing.

We are using the same Centos7 VM and installed

Install TwitterAPI

> sudo pip install TwitterAPI

Required App Configuration

You will need to make note of the following TwitterApp values:

"credential": 

{

"CONSUMER_KEY": "XXXXXXXXXX",

"CONSUMER_SECRET": "XXXXXXXXXXXXX",

"ACCESS_TOKEN_KEY": "XXXXXXXXXX",

"ACCESS_TOKEN_SECRET": "XXXXXXXXXX"

}

This will be used to authenticate against Twitter's Stream server(s). Please bare in mind that your IP will be banned if your app goes out of control :)

Stream connection function:

It is as simple as creating a TwitterAPI object by passing the above app configuration values.

def app_connect(app_config):

        try:

                return TwitterAPI(app_config_json['credential']['CONSUMER_KEY'],

                        app_config_json['credential']['CONSUMER_SECRET'],

                        app_config_json['credential']['ACCESS_TOKEN_KEY'],

                        app_config_json['credential']['ACCESS_TOKEN_SECRET'])

        except Exception as e:

                logging.debug("TwitterAPI Exception: %s" + str(e))

                exit(1)

TRACK - our first experiment 

A comma-separated list of phrases which will be used to determine what Tweets will be delivered on the stream. Our list of phrases is stored in a separate json config file that is loaded when the code is run.

{ 

"et": 

{

    "track": ["ethiopia","addisababa","ethioi","Oromo","Amhara","Somali","Tigray","Sidama","Gurage","Welayta","Hadiya","Afar","Gamo","Gedeo"]

},

"so": 

{

                "track": ["somalia","republicofsomalia","somaliland","puntland","galmudug","jubalandkhaatumo","koonfurgalibeed"]

        }

} 

Time to make the request

As per documentation, we call the request method to filter statuses

keep in mind the restrictions such as: Each phrase must be between 1 and 60 bytes, inclusive.

> req = twitter_conn.request('statuses/filter', {'track': TRACK_TERM})

Summary

This is all it takes to build a simple code that taps into Twitter's Stream to filter out Twits based on a comma separated strings.

Clone/hack/improve it as always - 

• git@gitlab.com:baricho/feed_twitter.git
• https://gitlab.com/baricho/feed_twitter.git

Peace!