Twitfinder

Today, I decided to do a quick coding project, although my time was limited. I ended up settling on starting a module to find things via Twitter. I took a quick peek at the API, and quickly found it too big a task to do something worthwhile while also still figuring out some of the basics of using nodejs or python to communicate with the web.

Recently, I had been working on a module to find the top words on sites using node js, so I decided to expand this to find the Twitter users which are currently talking about a specific keyword. I used the request and cheerio packages from npm, and combined them with some quick browsing of the Twitter page for a search to get started.

searchTerm = 'bike';
url = "https://twitter.com/searchq="+searchTerm+"&src=typd&vertical=default&f=tweets";
request(url, function (error, response, body) {
if (!error) {
    // load a page into cheerio
    // grab the entire body.
    var $page = cheerio.load(body);

    //.js-action-profile-name is the class that stores usernames
    //Let's grab all of them ~20 at a time and log to console.
    text = $page('.username.js-action-profile-name').text();

I then made the request, and (after some experimenting) found that .username.js-action-profile-name was the appropriate class to use for finding a list of the usernames. Using the cheerio function .text(), I was then able to create a string of usernames, which conveniently are all separated by an "@". I then split the usernames, and created some code for later usage when I log a much longer list of this data.

text.split("@").forEach(function(user){
        if (userList[user]) {
            userList[user]++;
        } else {
            userList[user] = 1;
            totalUsers++;
        }
    });

I enjoyed creating this little start to code, and plan to continue on (and use the API) to create a tool which might be useful to some people.