However, the situation I needed to fix was in this particular scenario we had a total of 6x Mac Caching servers across different sites. And while yes, you could absolutely get six individual charts for Each appliance this didn’t logically make sense - I was after more of an ”at a glance dashboard”
Map out requirements and goals
In this particular scenario, the Mac device at the most dense site would show the chart, and then all the other sites would be categorized underneath that chart with the last two data entry points of how much data has been served to clients however, this was a calculation between the two previous readings, so it was more of a real time calculation
I needed to add next to the data points an arrow that would either trend up or down based on whether the load was higher or lower.
Next, I needed to add a easy to glance view a state of whether the device was healthy or unhealthy - If the value served from the last two points was zero it would report as unhealthy, anything else it would get the healthy status.
The final requirement was if the device was unhealthy, I wanted a red flashing dot next to unhealthy status, however, is the device was healthy a green dot would be required.
That concludes all the requirements now we need to get down to creating it.
Understanding requirements fordate points
Just, we already have the demon set up that will log the data we require every hour. This is also based on the last value to show you how much data has being served.
This file is currently on macOS, however, because we have more than one device to manage. We will do the report Generation from Windows by using Powershell.
Password free script
Nobody ever wants to store passwords in scripts, and this is no exception, itwill use an XML file that will have the encrypted data to use to connect to SSH securely and pull the file required back to the Windows computer.
Generate your macOS credentials and create XML file
The first thing we need to do is create the encrypted credentials in an XML file that will be used to connect back your Mac OS the Files we used to the next phase with the website generation.
The commands Below will generate your XML file, Remember, these are the same login credentials you use to access your macOS device (Do not use your Windows credentials here)
$Credential = Get-Credential
$Credential | Export-Clixml -Path "credentials.xml"
Retrieve files from macOS device to Windows
We now need the script to retrieve the correct CSV file from the macOS device this will be placed in a folder designated by the script that will need to Windows.
Wendy’s files are copied in the specified directory using SSH. They will also be renamed based on the file in the variables at the top of the script so the same file is not overriding itself, This naming convention in my example is a site code, followed by an_ (underscore)
This script is therefore executed in Powershell (obviously on Windows) - and remember to update the variables in bold to match your environment!
Script: CopyCachingLogs.ps1
$PuttyPath = "C:\Program Files\PuTTY\pscp.exe"
$CredentialFile = "credentials.xml"
Remove-Item -Path "C:\MacReports\*" -Force -ErrorAction SilentlyContinue
# Import credentials from XML
$Credential = Import-Clixml -Path $CredentialFile
$Username = $Credential.UserName
$Password = $Credential.GetNetworkCredential().Password
# Define list of remote hosts and their corresponding filename prefixes
$Servers = @{
"10.78.299.1" = "hq_”
"10.76.299.1" = "nyc_"
"10.77.299.1" = "sfo_"
"10.78.299.1" = "lax_"
"10.79.299.1" = "atl_"
"10.80.299.1" = "mco_"
}
# List of files to copy with their paths
$"cache_stats.csv" = "/var/log/"
}
$LocalPath = "C:\MacReports\"
# Step 1: Delete existing local copies before downloading
foreach ($File in $Files.Keys) {
$LocalFile = "$LocalPath$File"
if (Test-Path $LocalFile) {
Write-Host "Deleting old local file: $LocalFile"
Remove-Item -Path $LocalFile -Force
}
}
# Step 2: Loop through each server and download files
foreach ($RemoteHost in $Servers.Keys) {
$Prefix = $Servers[$RemoteHost]
foreach ($File in $Files.Keys) {
$RemotePath = $Files[$File]
$LocalFile = "$LocalPath$File"
$RenamedFile = "$LocalPath$Prefix$File"
Write-Host "Transferring $File from $RemoteHost to $LocalFile..."
& $PuttyPath -q -scp -l $Username -pw $Password $RemoteHost`:$RemotePath$File $LocalFile
# Ensure the transfer was successful before renaming
if (Test-Path $LocalFile) {
Write-Host "Renaming $LocalFile to $RenamedFile"
Rename-Item -Path $LocalFile -NewName $RenamedFile -Force
} else {
Write-Host "ERROR: Transfer failed for $File from $RemoteHost"
}
}
}
Write-Host "All transfers completed successfully."
Check destination folder
If you check the destination folder, you should now now see all the server where you have requested SSH transfer the file to should now be in that folder with the site prefix before the name of the file
Reason we have a site code in my example, followed by an_ Is that character is going to tell the script that generates the website what to call that site - This means it’s easily identifiable at a glance.
I have also included in the script, the command to clear the folder before we transferred the files to make sure they are the latest copy of those files.
Create the HTML report
We now only have one thing left to do which is the process those files and create the HTML report.
The only customization you need to do with the script please make sure the folder path is correct.
Script : Generate-Report.ps1
# Configuration
$dataDir = "C:\MacReports"
$primarySite = "bearhq_cache_stats.CSV"
$files = Get-ChildItem -Path $dataDir -Filter "*cache_stats.csv"
function Process-CacheData {
param($filePath, $numLines = 30)
if (!(Test-Path $filePath)) {
Write-Warning "File not found: $filePath"
return $null
}
$data = Import-Csv $filePath | Select-Object -Last $numLines
if ($null -eq $data) {
Write-Warning "No data found in $filePath"
return $null
}
$dates = $data | ForEach-Object { "`"$($_.timestamp)`"" }
$toClients = $data | ForEach-Object {
if ($_.to_clients_since_last_gb -match "^\d*\.?\d*$") {
[double]$_.to_clients_since_last_gb
} else {
0
}
}
$fromOrigin = $data | ForEach-Object {
if ($_.from_origin_since_last_gb -match "^\d*\.?\d*$") {
[double]$_.from_origin_since_last_gb
} else {
0
}
}
return @{
dates = "[" + ($dates -join ",") + "]"
toClients = "[" + ($toClients -join ",") + "]"
fromOrigin = "[" + ($fromOrigin -join ",") + "]"
}
}
function Get-SiteHealth {
param($filePath)
if (!(Test-Path $filePath)) {
Write-Warning "File not found: $filePath"
return $null
}
$data = Import-Csv $filePath
if ($null -eq $data -or $data.Count -eq 0) {
Write-Warning "Insufficient data in $filePath"
return @{
site = ([System.IO.Path]::GetFileNameWithoutExtension($filePath)).Split('_')[0]
lastTwoValues = "0, 0"
trend = "-"
totalTransfer = 0
healthy = $false
}
}
$siteName = ([System.IO.Path]::GetFileNameWithoutExtension($filePath)).Split('_')[0]
# Get last two values
$lastTwo = $data | Select-Object -Last 2
$lastValues = @($lastTwo | ForEach-Object {
if ($_.to_clients_since_last_gb -match "^\d*\.?\d*$") {
[double]$_.to_clients_since_last_gb
} else {
0
}
})
# Calculate total transfer
$totalTransfer = ($data | ForEach-Object {
if ($_.to_clients_since_last_gb -match "^\d*\.?\d*$") {
[double]$_.to_clients_since_last_gb
} else {
0
}
} | Measure-Object -Sum).Sum
$valuesString = "$([math]::Round($lastValues[0], 2)), $([math]::Round($lastValues[1], 2))"
$trend = if ($lastValues.Count -gt 1 -and $lastValues[-1] -gt $lastValues[0]) { "↑" } else { "↓" }
# Site is healthy if either last two values or total transfer is non-zero
$isHealthy = ($lastValues[0] -gt 0) -or ($lastValues[1] -gt 0) -or ($totalTransfer -gt 0)
return @{
site = $siteName
lastTwoValues = $valuesString
trend = $trend
totalTransfer = [math]::Round($totalTransfer, 2)
healthy = $isHealthy
}
}
# Process main chart data (Primary Site)
Write-Host "Processing main chart data..."
$mainData = Process-CacheData "$dataDir\$primarySite"
if ($null -eq $mainData) {
Write-Error "Failed to process main chart data"
exit 1
}
# Process primary site health data first
$siteHealth = @()
$primaryHealth = Get-SiteHealth "$dataDir\$primarySite"
if ($null -ne $primaryHealth) {
$siteHealth += $primaryHealth
}
# Process other site health data
$files | Where-Object { $_.Name -ne $primarySite } | ForEach-Object {
$health = Get-SiteHealth $_.FullName
if ($null -ne $health) {
$siteHealth += $health
}
}
# Generate HTML
$healthRows = $siteHealth | ForEach-Object {
$healthStatus = if ($_.healthy) {
"<span class='health-status healthy'><span>●</span> Healthy</span>"
} else {
"<span class='health-status unhealthy'><span class='blink'>●</span> Unhealthy</span>"
}
"<div class='health-card'>
<div class='site-name'>$($_.site)</div>
<div class='metrics'>
<div class='metric'>
<div class='metric-label'>Last Two Transfers</div>
<div class='metric-value'>$($_.lastTwoValues) GB/hr $($_.trend)</div>
</div>
<div class='metric'>
<div class='metric-label'>Total Transfer</div>
<div class='metric-value'>$($_.totalTransfer) GB</div>
</div>
<div class='metric'>
<div class='metric-label'>Status</div>
<div class='metric-value'>$healthStatus</div>
</div>
</div>
</div>"
}
$html = @"
<!DOCTYPE html>
<html>
<head>
<title>Cache Statistics</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/3.9.1/chart.min.js"></script>
<style>
body {
font-family: -apple-system, BlinkMacSystemFont, sans-serif;
margin: 0;
padding: 20px;
background: #f5f5f7;
}
.container {
max-width: 1200px;
margin: 0 auto;
background: white;
padding: 20px;
border-radius: 10px;
box-shadow: 0 1px 3px rgba(0,0,0,0.1);
}
.chart-container {
width: 100%;
height: 300px;
position: relative;
margin-bottom: 20px;
}
.health-cards {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(300px, 1fr));
gap: 20px;
padding: 20px 0;
}
.health-card {
background: white;
border-radius: 10px;
padding: 20px;
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
border: 1px solid #e1e1e1;
min-width: 300px;
}
.site-name {
font-size: 1.2em;
font-weight: 600;
margin-bottom: 15px;
color: #1a1a1a;
padding-bottom: 10px;
border-bottom: 1px solid #eee;
}
.metrics {
display: flex;
flex-direction: column;
gap: 12px;
}
.metric {
display: flex;
justify-content: space-between;
align-items: center;
}
.metric-label {
color: #666;
font-size: 0.9em;
}
.metric-value {
font-weight: 500;
color: #1a1a1a;
}
.health-status {
display: inline-flex;
align-items: center;
gap: 5px;
padding: 4px 8px;
border-radius: 4px;
font-weight: 500;
}
.health-status.healthy {
background: rgba(0, 200, 0, 0.1);
color: green;
}
.health-status.unhealthy {
background: rgba(255, 0, 0, 0.1);
color: red;
}
@keyframes blink {
0% { opacity: 1; }
50% { opacity: 0; }
100% { opacity: 1; }
}
.blink {
animation: blink 1s infinite;
}
</style>
</head>
<body>
<div class="container">
<h1>Cache Statistics Report</h1>
<div class="chart-container">
<canvas id="myChart"></canvas>
</div>
<div class="health-cards">
$($healthRows -join "`n")
</div>
</div>
<script>
const chartData = {
labels: $($mainData.dates),
datasets: [
{
label: 'To Clients (GB/hr)',
data: $($mainData.toClients),
borderColor: '#007AFF',
backgroundColor: 'rgba(0, 122, 255, 0.1)',
tension: 0.1
},
{
label: 'From Origin (GB/hr)',
data: $($mainData.fromOrigin),
borderColor: '#FF3B30',
backgroundColor: 'rgba(255, 59, 48, 0.1)',
tension: 0.1
}
]
};
const ctx = document.getElementById('myChart');
new Chart(ctx, {
type: 'line',
data: chartData,
options: {
responsive: true,
maintainAspectRatio: false,
plugins: {
title: {
display: true,
text: 'Primary Site Cache Transfer Statistics'
}
},
scales: {
y: {
beginAtZero: true,
title: {
display: true,
text: 'Data Transfer (GB/hr)'
}
},
x: {
ticks: {
maxRotation: 45,
minRotation: 45
}
}
}
}
});
</script>
</body>
</html>
"@
$html | Out-File "cache_report.html" -Encoding UTF8
Write-Host "Report generated as cache_report.html"