Wednesday, September 19, 2018

non-API tweet processing

I wrote some data.table based code  in R 3.51  that allowed me to examine 6 months of my tweets without use of an API. This means average users can look at their own tweets without creating an application, authenticating, etc (See 1, 2, 3) . First I downloaded six months of tweets and renamed them by month:

tw201804.csv
tw201805.csv
tw201806.csv
tw201807.csv
tw201808.csv
tw201809.csv

The code far below row binds those six months together, removes the spaces between the column names, then chooses select columns with some configuration. The lattice library gives me the charts below.  Binding together the metrics with time as a factor (~ likes + urlclicks + engagements | as.factor(time))  consolidates tweet metrics by day. Click to enlarge charts:



# you will need install.packages() for these libraries
library(bit64)
library(data.table)
library(lubridate)
library(lattice)

tw <- {}; for(i in 4:9) {tw <- rbind(tw,fread(paste0("C:\\Users\\rferrisx\\Downloads\\tw20180",i,".csv")))}
names(tw) <- tw[,gsub(" ","",names(tw))]
tw20180409 <- tw[,.(tweet=substr(Tweettext,1,40),
time=ymd(substr(time,0,10)),
impressions,
likes,
engagements,
engagePCT=round(engagementrate,3) * 100,
retweets,
replies,
urlclicks,
detailexpands)]

tw20180409[,Index:=.I]
tw20180409[,barchart(~ likes + urlclicks + engagements | as.factor(Index),allow.multiple=TRUE,origin=0,data=tw20180409,auto.key=list(rev=TRUE,reverse.rows=TRUE))]
tw20180409[,barchart(~ likes + urlclicks + engagements | as.factor(time),allow.multiple=TRUE,origin=0,data=tw20180409,auto.key=list(rev=TRUE,reverse.rows=TRUE))]
tw20180409[,barchart(~ likes + urlclicks + engagements | as.factor(tweet),allow.multiple=TRUE,origin=0,data=tw20180409,auto.key=list(rev=TRUE,reverse.rows=TRUE))]
tw20180409[,barchart(~ impressions | as.factor(time),allow.multiple=TRUE,origin=0,data=tw20180409,auto.key=list(rev=TRUE,reverse.rows=TRUE))]
tw20180409[,barchart(~ impressions | as.factor(Index),allow.multiple=TRUE,origin=0,data=tw20180409,auto.key=list(rev=TRUE,reverse.rows=TRUE))]