non-API tweet processing

I wrote some data.table based code  in R 3.51  that allowed me to examine 6 months of my tweets without use of an API. This means average users can look at their own tweets without creating an application, authenticating, etc (See 1, 2, 3) . First I downloaded six months of tweets and renamed them by month:

tw201804.csv
tw201805.csv
tw201806.csv
tw201807.csv
tw201808.csv
tw201809.csv

The code far below row binds those six months together, removes the spaces between the column names, then chooses select columns with some configuration. The lattice library gives me the charts below.  Binding together the metrics with time as a factor (~ likes + urlclicks + engagements | as.factor(time))  consolidates tweet metrics by day. Click to enlarge charts:

# you will need install.packages() for these libraries
library(bit64)
library(data.table)
library(lubridate)
library(lattice)

tw <- {}; for(i in 4:9) {tw <- rbind(tw,fread(paste0("C:\\Users\\rferrisx\\Downloads\\tw20180",i,".csv")))}
names(tw) <- tw[,gsub(" ","",names(tw))]
tw20180409 <- tw[,.(tweet=substr(Tweettext,1,40),
time=ymd(substr(time,0,10)),
impressions,
likes,
engagements,
engagePCT=round(engagementrate,3) * 100,
retweets,
replies,
urlclicks,
detailexpands)]

tw20180409[,Index:=.I]
tw20180409[,barchart(~ likes + urlclicks + engagements | as.factor(Index),allow.multiple=TRUE,origin=0,data=tw20180409,auto.key=list(rev=TRUE,reverse.rows=TRUE))]
tw20180409[,barchart(~ likes + urlclicks + engagements | as.factor(time),allow.multiple=TRUE,origin=0,data=tw20180409,auto.key=list(rev=TRUE,reverse.rows=TRUE))]
tw20180409[,barchart(~ likes + urlclicks + engagements | as.factor(tweet),allow.multiple=TRUE,origin=0,data=tw20180409,auto.key=list(rev=TRUE,reverse.rows=TRUE))]
tw20180409[,barchart(~ impressions | as.factor(time),allow.multiple=TRUE,origin=0,data=tw20180409,auto.key=list(rev=TRUE,reverse.rows=TRUE))]
tw20180409[,barchart(~ impressions | as.factor(Index),allow.multiple=TRUE,origin=0,data=tw20180409,auto.key=list(rev=TRUE,reverse.rows=TRUE))]

11.71875

A

## Ryan Matthew Ferris 10:26 PM Thursday, November 19, 2015, 2015 Bellingham, WA

## Updated 7/10/2018

## *Notes on 187.5/16 or 11.71875*

##

# In this code, I introduce 'xi'  (11.71875) a constant with strong relationships to both pi and # exp(1).

# 'Xi' (11.71875 or (187.5/16)) is a 'translator constant' between powers of 2 and 16.

# Search google for "11.71875mhz" and you can see this. 11.71875mhz is a widely used frequency in 'crystal resonators'.

# More than simply being an efficient means to compute pi or exp(1) (e.g. 'Eulers number'), 'xi' may

# 'interpolate' between pi and e, giving science the means to measure logarithmic growth patterns of

# increasing radii. This may make it useful for gravitational and rotational calculations. It is

# possible that the 187.5 cm-3 pc represents some fixed attenuation limit  (see

# http://arxiv.org/abs/1503.05245) for radio waves signal that require either boosting or

# chirping.  The relationships between pi, e, and xi(187.5./16 = 11.71875) may represent fixed

# constants descriptive of all  EM signaling. In addition, the constants that describe the

# circumscription of an octahedron (sqrt(1/2) and sqrt(1/6) can also be used to derive xi with a

# factor close to 9/16.

 

## A number of constants and function names are used. Some are reused:

# [1] "almost2" "aR"      "c2"      "cR"      "eR"      "F_pe"    "F_pi"    "findpi"  "i"       "j"       "k"       "k1"    

# [13] "k2"      "l"       "l1"      "mm1"     "p1"      "p2"      "piR"     "pR"      "pX"      "s4"      "seqe"    "seqpi" 

# [25] "seqpi_e" "t1"      "t2"      "t3"      "Vx"      "x"       "xi"      "xi_sqrt" "xif"     "y"     

## Set your graphics device to record history if desired. 86 graphs are produced.

## You may want par(mfrow=c(1,2))

## NB : In R 3.5 there is no bigint by default,however there are 22 digits of accuracy.

## See my usage of BIGINT in Powershell

## For this work I used Microsoft Open R 3.5 with Intel's MKL (Math Kernel Library)

Calling Powershell Memory Management Metrics inside R

There are a lot of memory management and memory management metrics in Windows 10 e.g (Resource monitor, performance monitor, kernel logging in event viewer) .  However, you can call Powershell memory management functions and metrics from inside R. This allows you to measure how Windows sees your libraries, functions, etc. from inside your R scripts. Windows 10 memory management uses "memory compression". "Memory Compression" is a 'hidden process' in PS. You can mix R's memory commands (e.g. memory.size,memory.limt,gc) and call cmd shell (eg. 'shell('systeminfo | findstr "Memory"')' ). [This post needs viewing on a wide screen.]