Calling Powershell Memory Management Metrics inside R

There are a lot of memory management and memory management metrics in Windows 10 e.g (Resource monitor, performance monitor, kernel logging in event viewer) .  However, you can call Powershell memory management functions and metrics from inside R. This allows you to measure how Windows sees your libraries, functions, etc. from inside your R scripts. Windows 10 memory management uses "memory compression". "Memory Compression" is a 'hidden process' in PS. You can mix R's memory commands (e.g. memory.size,memory.limt,gc) and call cmd shell (eg. 'shell('systeminfo | findstr "Memory"')' ). [This post needs viewing on a wide screen.]

> gcinfo(verbose=TRUE)
[1] FALSE
> gcinfo(verbose=TRUE)
[1] TRUE
> gc()
          used (Mb) gc trigger (Mb) max used (Mb)
Ncells  579063 31.0    1613298 86.2  1572014   84
Vcells 1233956  9.5    8388608 64.0 19382396  148

> gc(verbose=TRUE)
Garbage collection 4252 = 2635+1100+517 (level 2) ... 
31.0 Mbytes of cons cells used (36%)
9.5 Mbytes of vectors used (15%)
          used (Mb) gc trigger (Mb) max used (Mb)
Ncells  579072 31.0    1613298 86.2  1572014   84
Vcells 1233985  9.5    8388608 64.0 19382396  148

> memory()
Total Physical Memory:     16,269 MB
Available Physical Memory: 9,722 MB
Virtual Memory: Max Size:  24,403 MB
Virtual Memory: Available: 14,597 MB
Virtual Memory: In Use:    9,806 MB
     [,1]                                      
[1,] "0"                                       
[2,] "R Memory size (malloc) available: 243 MB"
[3,] "R Memory size (malloc) in use: 65.16 MB" 
[4,] "R Memory limit (total alloc): 16268 MB"  


But Powershell gives you a look at  metrics inside an R script.

# utils::setWindowTitle(x) # for multiple sessions
# Powershell memory counters from admin PS 5.1


PS C:\WINDOWS\system32> ps | gm

....

Id                         Property       int Id {get;}
MachineName                Property       string MachineName {get;}
MainModule                 Property       System.Diagnostics.ProcessModule MainModule {get;}
MainWindowHandle           Property       System.IntPtr MainWindowHandle {get;}
MainWindowTitle            Property       string MainWindowTitle {get;}
MaxWorkingSet              Property       System.IntPtr MaxWorkingSet {get;set;}
MinWorkingSet              Property       System.IntPtr MinWorkingSet {get;set;}
Modules                    Property       System.Diagnostics.ProcessModuleCollection Modules {get;}
NonpagedSystemMemorySize   Property       int NonpagedSystemMemorySize {get;}
NonpagedSystemMemorySize64 Property       long NonpagedSystemMemorySize64 {get;}
PagedMemorySize            Property       int PagedMemorySize {get;}
PagedMemorySize64          Property       long PagedMemorySize64 {get;}
PagedSystemMemorySize      Property       int PagedSystemMemorySize {get;}
PagedSystemMemorySize64    Property       long PagedSystemMemorySize64 {get;}
PeakPagedMemorySize        Property       int PeakPagedMemorySize {get;}
PeakPagedMemorySize64      Property       long PeakPagedMemorySize64 {get;}
PeakVirtualMemorySize      Property       int PeakVirtualMemorySize {get;}
PeakVirtualMemorySize64    Property       long PeakVirtualMemorySize64 {get;}
PeakWorkingSet             Property       int PeakWorkingSet {get;}
PeakWorkingSet64           Property       long PeakWorkingSet64 {get;}
PriorityBoostEnabled       Property       bool PriorityBoostEnabled {get;set;}
PriorityClass              Property       System.Diagnostics.ProcessPriorityClass PriorityClass {get;set;}
PrivateMemorySize          Property       int PrivateMemorySize {get;}
PrivateMemorySize64        Property       long PrivateMemorySize64 {get;}
PrivilegedProcessorTime    Property       timespan PrivilegedProcessorTime {get;}
ProcessName                Property       string ProcessName {get;}

....


# Powershell functions called from R functions on Windows 10; 
# Note the inclusion of 'hidden process' "Memory Compression"


ps <- function() { system("powershell -ExecutionPolicy Bypass -command ps -name RGui | ft MainWindowTitle,Id,HandleCount,WorkingSet64,PeakWorkingSet64,PrivateMemorySize64,VirtualMemorySize64 "); }

ps_e <- function() { system("powershell -ExecutionPolicy Bypass -command ps | where {$_.Name -EQ 'RGui' -or $_.Name -EQ 'Memory Compression'} | ft Name,Id,HandleCount,WorkingSet64,PeakWorkingSet64,PrivateMemorySize64,VirtualMemorySize64 "); }

ps_f <- function() { system("powershell -ExecutionPolicy Bypass -command $t1 = ps | where {$_.Name -EQ 'RGui' -or $_.Name -EQ 'Memory Compression'};
$t2 = $t1 | Select { 
 $_.Id;
 [math]::Round($_.WorkingSet64/1MB);
 [math]::Round($_.PrivateMemorySize64/1MB);
 [math]::Round($_.VirtualMemorySize64/1MB) };
$t2 | ft * "); }

ps_all <- function() {ps();ps_e();ps_f();}

memory <- function() {
as.matrix(list(
paste0(shell('systeminfo | findstr "Memory"')), # Windows cmd shell function
paste0("R Memory size (malloc) available: ",memory.size(TRUE)," MB"),
paste0("R Memory size (malloc) in use: ",memory.size()," MB"),
paste0("R Memory limit (total alloc): ",memory.limit()," MB")
))
}


# Output
# Enumerated in Bytes unless otherwise noted

> ps()

MainWindowTitle                                                Id HandleCount WorkingSet64 PeakWorkingSet64 PrivateMemo
                                                                                                      rySize64

RGui  -  Microsoft R Open version 3.5 (64-bit) - [R Console] 2908         336     16670720        296427520   298504192


> ps_e()

Name                 Id HandleCount WorkingSet64 PeakWorkingSet64 PrivateMemorySize64 VirtualMemorySize64
----                 -- ----------- ------------ ---------------- ------------------- -------------------
Memory Compression 1880           0    349683712        577429504             1085440           392036352
Rgui               2908         336     16748544        296427520           298504192          5142429696


> ps_f()

                                                                                                                       
 $_.Id;                                                                                                                
 [math]::Round($_.WorkingSet64/1MB);                                                                                   
 [math]::Round($_.PrivateMemorySize64/1MB);                                                                            
 [math]::Round($_.VirtualMemorySize64/1MB)                                                                             
{1880, 333, 1, 374}                                                                                                    
{2908, 16, 285, 4904}                                                                                                  


> memory()
Total Physical Memory:     16,269 MB
Available Physical Memory: 9,979 MB
Virtual Memory: Max Size:  24,403 MB
Virtual Memory: Available: 14,793 MB
Virtual Memory: In Use:    9,610 MB
     [,1]                                      
[1,] "0"                                       
[2,] "R Memory size (malloc) available: 243 MB"
[3,] "R Memory size (malloc) in use: 133.2 MB" 
[4,] "R Memory limit (total alloc): 16268 MB"