Powershell : Performance counter Best Practices Analyzer

If you are running Windows Servers then complying with the relevant performance counters and best practices can make your experience on that server painless and fluid.


Performance counters to analyze

let’s start with each performance counter, what it means, the importance of that performance counter, recommended values where it should be and then finally why it matters!

Processor Time (% Processor Time)

   - What it measures: Percentage of time the processor spends executing non-idle threads

   - Importance: Primary indicator of CPU utilization and processing power consumption

   - Recommended values:

     - Healthy: < 70%
    - Warning: 70-90%
     - Critical: > 90%

   - Why it matters: Sustained high CPU usage can indicate processing bottlenecks, insufficient CPU resources, or problematic applications

Processor Queue Length

   - What it measures: Number of threads waiting for processor time

   - Importance: Indicates if there are more threads ready to execute than available CPU cores

   - Recommended values:

     - Healthy: < 2
     - Warning: 2-5
     - Critical: > 5

   - Why it matters: A sustained queue length above 2 per processor suggests CPU contention and potential performance bottlenecks

Disk Queue Length (Current Disk Queue Length)

   - What it measures: Number of I/O requests queued for disk access

   - Importance: Key indicator of disk subsystem performance

   - Recommended values:

     - Healthy: < 2 per spindle
     - Warning: 2-5
     - Critical: > 5

   - Why it matters: High queue lengths indicate disk bottlenecks that can slow down all system operations

Disk Read Time (Avg. Disk sec/Read)

   - What it measures: Average time in milliseconds for read operations

   - Importance: Indicates disk read performance

   - Recommended values:

     - Healthy: < 15ms
     - Warning: 15-25ms
     - Critical: > 25ms

   - Why it matters: High read times can significantly impact application performance and system responsiveness

Disk Write Time (Avg. Disk sec/Write)

   - What it measures: Average time in milliseconds for write operations

   - Importance: Indicates disk write performance

   - Recommended values:

     - Healthy: < 15ms
     - Warning: 15-25ms
     - Critical: > 25ms

   - Why it matters: Slow write times can cause application delays and system performance issues

Pages per Second

   - What it measures: Rate at which pages are read from or written to disk for virtual memory operations

   - Importance: Key indicator of memory pressure and paging activity

   - Recommended values:

     - Healthy: < 1000/sec
     - Warning: 1000-2000/sec
     - Critical: > 2000/sec

   - Why it matters: High paging rates indicate insufficient physical memory and can severely impact system performance

Available Memory (Available MBytes)

   - What it measures: Amount of physical memory available for allocation

   - Importance: Critical indicator of memory resources

   - Recommended values:

     - Healthy: > 512MB
     - Warning: 256-512MB
     - Critical: < 256MB

   - Why it matters: Low available memory can lead to excessive paging and system performance degradation

Context Switches per Second

   - What it measures: Rate at which the processor switches from one thread to another

   - Importance: Indicates how often the processor needs to save and restore thread states

   - Recommended values:

     - Healthy: < 15,000/sec
     - Warning: 15,000-20,000/sec
     - Critical: > 20,000/sec

   - Why it matters: Excessive context switching can indicate thread management issues, CPU contention, or poorly designed applications

Best Practices for Monitoring

  1. Monitor these counters over time to establish baseline performance
  2. Look for patterns and trends rather than isolated spikes
  3. Consider the relationship between counters (e.g., high CPU + high queue length = CPU bottleneck)

Adjust thresholds based on your specific environment and requirements

Warning : When optimizing performance counts simply do not put the values incredibly high that they report everything is healthy - they need to be realistic values!

Remember that these thresholds are general guidelines and may need adjustment based on:

  1. Server roles and functions
  2. Hardware specifications
  3. Application requirements
  4. Time of day/peak usage periods
  5. Business requirements

Collecting the data

When collecting the data can you do it locally on the server, Alternatively, you can do it remotely from another server, However, if you’re looking to do it remotely from another server, you need to have the correct file access to be able to connect to get the data.

You can quite easily customize the script to get remote data however, the script that I have coded will not only get you the raw values every sample, It will also aggregate them into an average in a single text file with the values average out.

You will get an option in the script, which we will see later, that gives you how many samples to get with the delta between those samples in seconds - as an example, the following logic can be applied:

50 samples, 1 second apart for 50 seconds of data
50 sample, 3 seconds apart for 150 seconds of data

Script : Local_ServerPerformance.ps1

# Performance counters to monitor

$counters = @(
    "\Processor(_Total)\% Processor Time",
    "\System\Processor Queue Length",
    "\PhysicalDisk(_Total)\Current Disk Queue Length",
    "\PhysicalDisk(_Total)\Avg. Disk sec/Read",
    "\PhysicalDisk(_Total)\Avg. Disk sec/Write",
    "\Network Interface(*)\Output Queue Length",
    "\Memory\Available MBytes",
    "\Memory\Pages/sec",
    "\System\Context Switches/sec"
)

# Initialize array to store results
$results = @()

# Sampling parameters
$samples = 20    # Number of consecutive samples to collect
$sampleInterval = 1    # Wait between each sample

Write-Host "Starting local performance monitoring for $samples consecutive samples..."

for ($i = 1; $i -le $samples; $i++) {
    $timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss.fff"
    Write-Host "Taking sample $i of $samples at $timestamp" -NoNewline
    
    try {
        # Get counter values
        $counterValues = Get-Counter -Counter $counters -ErrorAction Stop
        
        # Create custom object for results
        $resultObject = [PSCustomObject]@{
            Timestamp = $timestamp
            ComputerName = $env:COMPUTERNAME
            SampleNumber = $i
            ProcessorTime = [math]::Round(($counterValues.CounterSamples | Where-Object {$_.Path -like "*\% Processor Time"}).CookedValue, 2)
            ProcessorQueueLength = ($counterValues.CounterSamples | Where-Object {$_.Path -like "*\Processor Queue Length"}).CookedValue
            DiskQueueLength = ($counterValues.CounterSamples | Where-Object {$_.Path -like "*\Current Disk Queue Length"}).CookedValue
            DiskReadTime = [math]::Round(($counterValues.CounterSamples | Where-Object {$_.Path -like "*\Avg. Disk sec/Read"}).CookedValue * 1000, 2)  # Convert to milliseconds
            DiskWriteTime = [math]::Round(($counterValues.CounterSamples | Where-Object {$_.Path -like "*\Avg. Disk sec/Write"}).CookedValue * 1000, 2)  # Convert to milliseconds
            NetworkQueueLength = ($counterValues.CounterSamples | Where-Object {$_.Path -like "*\Output Queue Length"}).CookedValue
            AvailableMemoryMB = ($counterValues.CounterSamples | Where-Object {$_.Path -like "*\Available MBytes"}).CookedValue
            PagesPerSec = ($counterValues.CounterSamples | Where-Object {$_.Path -like "*\Pages/sec"}).CookedValue
            ContextSwitchesPerSec = ($counterValues.CounterSamples | Where-Object {$_.Path -like "*\Context Switches/sec"}).CookedValue
            Status = "Success"
        }
        
        $results += $resultObject
        Write-Host " - Success"
    }
    catch {
        # Create error object if collection fails
        $resultObject = [PSCustomObject]@{
            Timestamp = $timestamp
            ComputerName = $env:COMPUTERNAME
            SampleNumber = $i
            ProcessorTime = $null
            ProcessorQueueLength = $null
            DiskQueueLength = $null
            DiskReadTime = $null
            DiskWriteTime = $null
            NetworkQueueLength = $null
            AvailableMemoryMB = $null
            PagesPerSec = $null
            ContextSwitchesPerSec = $null
            Status = "Error: $($_.Exception.Message)"
        }
        
        $results += $resultObject
        Write-Host " - Failed: $($_.Exception.Message)"
    }
    
    # Wait for next sample interval if not the last sample
    if ($i -lt $samples) {
        Start-Sleep -Seconds $sampleInterval
    }
}

# Calculate averages
$summaryResults = $results | 
    Where-Object {$_.Status -eq "Success"} |
    Group-Object ComputerName | 
    ForEach-Object {
        [PSCustomObject]@{
            ComputerName = $_.Name
            AvgProcessorTime = [math]::Round(($_.Group | Measure-Object -Property ProcessorTime -Average).Average, 2)
            AvgProcessorQueueLength = [math]::Round(($_.Group | Measure-Object -Property ProcessorQueueLength -Average).Average, 2)
            AvgDiskQueueLength = [math]::Round(($_.Group | Measure-Object -Property DiskQueueLength -Average).Average, 2)
            AvgDiskReadTime = [math]::Round(($_.Group | Measure-Object -Property DiskReadTime -Average).Average, 2)
            AvgDiskWriteTime = [math]::Round(($_.Group | Measure-Object -Property DiskWriteTime -Average).Average, 2)
            AvgNetworkQueueLength = [math]::Round(($_.Group | Measure-Object -Property NetworkQueueLength -Average).Average, 2)
            AvgAvailableMemoryMB = [math]::Round(($_.Group | Measure-Object -Property AvailableMemoryMB -Average).Average, 2)
            AvgPagesPerSec = [math]::Round(($_.Group | Measure-Object -Property PagesPerSec -Average).Average, 2)
            AvgContextSwitchesPerSec = [math]::Round(($_.Group | Measure-Object -Property ContextSwitchesPerSec -Average).Average, 2)
            SampleCount = ($_.Group | Measure-Object).Count
        }
    }

# Export detailed results to CSV
$detailedOutputPath = "PerformanceMetrics_Detailed_$(Get-Date -Format 'yyyyMMdd_HHmmss').csv"
$results | Export-Csv -Path $detailedOutputPath -NoTypeInformation

# Export summary results to CSV
$summaryOutputPath = "PerformanceMetrics_Summary_$(Get-Date -Format 'yyyyMMdd_HHmmss').csv"
$summaryResults | Export-Csv -Path $summaryOutputPath -NoTypeInformation

Write-Host "`nPerformance data collection completed."
Write-Host "Detailed results saved to: $detailedOutputPath"
Write-Host "Summary results saved to: $summaryOutputPath"

# Display summary results in console
Write-Host "`nSummary Results:"
$summaryResults | Format-Table -AutoSize

This will then give you a  CSV file with the headers that mirror the following values with the data points listed in that format as below:

"Timestamp","ComputerName","SampleNumber","ProcessorTime","ProcessorQueueLength","DiskQueueLength","DiskReadTime","DiskWriteTime","NetworkQueueLength","AvailableMemoryMB","PagesPerSec","ContextSwitchesPerSec","Status" 
"2025-02-17 10:50:25.356","BearWrkStation1","1","46.9","7","0","0","0.54","System.Object[]","58329","0.96757978783878","297808.480159535","Success"
"2025-02-17 10:50:32.044","BearWrkStation1","2","88.51","78","0","0","0.64","System.Object[]","58339","33.7646656825392","348570.00852606","Success"
"2025-02-17 10:50:36.371","BearWrkStation1","3","94.55","77","0","0.4","0.48","System.Object[]","58388","125.15531292968","587838.138366708","Success"

This file will be stored to the folder where the script is executed the script will confirm this when complete with:

Detailed results saved to: PerformanceMetrics_Detailed_20250217_154551.csv
Summary results saved to: PerformanceMetrics_Summary_20250217_154551.csv

This then look like this:


The files are formatted that the summary file is the "averaged values" and the "detailed" file will be the individual values from the script run earlier.

We now need to move the "detailed" file to a subfolder called WorkData that can be done with this script:

Script : MoveWorkingData.ps1

# Create WorkData folder if it doesn't exist
$workDataPath = ".\WorkData"
if (-not (Test-Path -Path $workDataPath)) {
    New-Item -Path $workDataPath -ItemType Directory
    Write-Host "Created WorkData directory"
}

# Move all files containing 'detailed' to WorkData folder
Get-ChildItem -Path "." -Filter "*detailed*.csv" | ForEach-Object {
    Move-Item -Path $_.FullName -Destination $workDataPath
    Write-Host "Moved $($_.Name) to WorkData folder"
}

Write-Host "File move operation completed"

Produce the HTML report (with Health Cards)

Finally we need to produce a HTML report based on the data in the "WorkData" folder this will analyse all the data, perform the averages itself and display a health card style website on that data like this:


This then shows us that the Context Switches and Processor variables are massively out of line and not healthy at all and should be investigated, then if you click the "Tootle Detailed Samples" you get all those sample individually at the times they were taken:


The correct health report should look more like this example:


These samples will also be colour coded to match the aesthetic of the reporting style.

Script : Generate_Report.ps1

# Define performance thresholds
$thresholds = @{
    ProcessorTime = @{
        Warning = 70
        Critical = 90
        Description = "Average CPU utilization should be below 70%"
    }
    ProcessorQueueLength = @{
        Warning = 2
        Critical = 5
        Description = "Should be less than 2 sustained"
    }
    DiskQueueLength = @{
        Warning = 2
        Critical = 5
        Description = "Should be less than 2 per spindle"
    }
    DiskReadTime = @{
        Warning = 15
        Critical = 25
        Description = "Should be under 15ms"
    }
    DiskWriteTime = @{
        Warning = 15
        Critical = 25
        Description = "Should be under 15ms"
    }
    PagesPerSec = @{
        Warning = 1000
        Critical = 2000
        Description = "Should be under 1000/sec"
    }
    AvailableMemoryMB = @{
        Warning = 512
        Critical = 256
        Description = "Should be above 512MB"
    }
    ContextSwitchesPerSec = @{
        Warning = 15000
        Critical = 20000
        Description = "Should be under 15000/sec"
    }
}

# Function to get health status
function Get-MetricHealth {
    param (
        [string]$MetricName,
        [double]$Value
    )
    
    if ($MetricName -eq 'AvailableMemoryMB') {
        if ($Value -lt $thresholds[$MetricName].Critical) { return 'Critical' }
        elseif ($Value -lt $thresholds[$MetricName].Warning) { return 'Warning' }
        else { return 'Healthy' }
    }
    else {
        if ($Value -gt $thresholds[$MetricName].Critical) { return 'Critical' }
        elseif ($Value -gt $thresholds[$MetricName].Warning) { return 'Warning' }
        else { return 'Healthy' }
    }
}

# Check if WorkData folder exists
$workDataPath = ".\WorkData"
if (-not (Test-Path -Path $workDataPath)) {
    Write-Error "WorkData folder not found!"
    exit
}

# Get all detailed CSV files
$csvFiles = Get-ChildItem -Path $workDataPath -Filter "*Detailed*.csv"
if ($csvFiles.Count -eq 0) {
    Write-Error "No detailed CSV files found in WorkData folder!"
    exit
}

# Initialize results
$serverResults = @{}

# Process each CSV file
foreach ($file in $csvFiles) {
    Write-Host "Processing $($file.Name)..."
    
    # Import CSV content
    $csvData = Import-Csv -Path $file.FullName
    
    # Process each computer's data
    $csvData | Group-Object -Property ComputerName | ForEach-Object {
        $computer = $_.Name
        $samples = $_.Group | Where-Object { $_.Status -eq "Success" }
        
        if ($samples.Count -eq 0) { return }
        
        if (-not $serverResults.ContainsKey($computer)) {
            $serverResults[$computer] = @{
                Summary = $null
                Details = @()
            }
        }
        
        # Calculate averages for summary
        $averages = @{
            ComputerName = $computer
            ProcessorTime = [math]::Round(($samples.ProcessorTime | Measure-Object -Average).Average, 2)
            ProcessorQueueLength = [math]::Round(($samples.ProcessorQueueLength | Measure-Object -Average).Average, 2)
            DiskQueueLength = [math]::Round(($samples.DiskQueueLength | Measure-Object -Average).Average, 2)
            DiskReadTime = [math]::Round(($samples.DiskReadTime | Measure-Object -Average).Average, 2)
            DiskWriteTime = [math]::Round(($samples.DiskWriteTime | Measure-Object -Average).Average, 2)
            PagesPerSec = [math]::Round(($samples.PagesPerSec | Measure-Object -Average).Average, 2)
            AvailableMemoryMB = [math]::Round(($samples.AvailableMemoryMB | Measure-Object -Average).Average, 2)
            ContextSwitchesPerSec = [math]::Round(($samples.ContextSwitchesPerSec | Measure-Object -Average).Average, 2)
            SampleCount = $samples.Count
            LastSample = ($samples | Select-Object -Last 1).Timestamp
        }
        
        # Add health status for summary
        $health = @{}
        foreach ($metric in $thresholds.Keys) {
            $health[$metric] = Get-MetricHealth -MetricName $metric -Value $averages[$metric]
        }
        $averages.Health = $health
        
        # Store summary
        $serverResults[$computer].Summary = $averages
        
        # Process individual samples
        $samples | ForEach-Object {
            $sampleHealth = @{}
            foreach ($metric in $thresholds.Keys) {
                $sampleHealth[$metric] = Get-MetricHealth -MetricName $metric -Value $_.$metric
            }
            
            $sampleData = @{
                Timestamp = $_.Timestamp
                Metrics = $_
                Health = $sampleHealth
            }
            
            $serverResults[$computer].Details += $sampleData
        }
    }
}

# Generate HTML content
$htmlHeader = @"
<!DOCTYPE html>
<html>
<head>
    <title>Server Performance Health Dashboard</title>
    <style>
        body { font-family: Arial, sans-serif; margin: 20px; background: #f5f5f5; }
        .server-section { background: white; padding: 20px; margin-bottom: 30px; border-radius: 8px; box-shadow: 0 2px 4px rgba(0,0,0,0.1); }
        .metric { margin: 10px 0; padding: 10px; background: #f8f9fa; border-radius: 4px; }
        .Healthy { color: #198754; background: #d1e7dd; padding: 3px 8px; border-radius: 3px; }
        .Warning { color: #856404; background: #fff3cd; padding: 3px 8px; border-radius: 3px; }
        .Critical { color: #721c24; background: #f8d7da; padding: 3px 8px; border-radius: 3px; }
        .server-name { font-size: 1.4em; font-weight: bold; margin-bottom: 15px; }
        .timestamp { color: #666; font-size: 0.9em; margin-bottom: 15px; }
        .metric-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(300px, 1fr)); gap: 15px; }
        .metric-name { font-weight: bold; }
        .metric-value { margin: 5px 0; }
        .metric-desc { font-size: 0.9em; color: #666; margin-top: 5px; }
        .details-section { margin-top: 20px; }
        .details-header { font-size: 1.1em; font-weight: bold; margin: 20px 0 10px 0; }
        table { width: 100%; border-collapse: collapse; margin-top: 10px; }
        th, td { padding: 8px; text-align: left; border-bottom: 1px solid #ddd; }
        th { background-color: #f8f9fa; }
        tr:hover { background-color: #f5f5f5; }
        .summary-card { background: #f8f9fa; padding: 15px; border-radius: 6px; margin-bottom: 20px; }
        .toggle-details { cursor: pointer; color: #0066cc; text-decoration: underline; }
    </style>
    <script>
        function toggleDetails(serverId) {
            var details = document.getElementById(serverId + '-details');
            if (details.style.display === 'none') {
                details.style.display = 'block';
            } else {
                details.style.display = 'none';
            }
        }
    </script>
</head>
<body>
    <h1>Server Performance Health Dashboard</h1>
"@

$htmlContent = foreach ($serverName in $serverResults.Keys) {
    $server = $serverResults[$serverName]
    $summary = $server.Summary
    $serverId = $serverName.Replace(".", "-")
    
    @"
    <div class="server-section">
        <div class="server-name">$serverName</div>
        <div class="timestamp">Last Sample: $($summary.LastSample)</div>
        
        <div class="summary-card">
            <h2>Summary (Averages)</h2>
            <div class="metric-grid">
"@
    
    foreach ($metric in $thresholds.Keys) {
        $value = $summary.$metric
        $health = $summary.Health[$metric]
        $description = $thresholds[$metric].Description
        
        @"
                <div class="metric">
                    <div class="metric-name">$metric</div>
                    <div class="metric-value">$value</div>
                    <span class="$health">$health</span>
                    <div class="metric-desc">$description</div>
                </div>
"@
    }
    
    @"
            </div>
        </div>
        
        <div class="toggle-details" onclick="toggleDetails('$serverId')">Toggle Detailed Samples</div>
        
        <div id="$serverId-details" class="details-section" style="display: none;">
            <h3>Detailed Samples</h3>
            <table>
                <tr>
                    <th>Timestamp</th>
"@
    
    foreach ($metric in $thresholds.Keys) {
        @"
                    <th>$metric</th>
"@
    }
    
    @"
                </tr>
"@
    
    foreach ($sample in $server.Details) {
        @"
                <tr>
                    <td>$($sample.Timestamp)</td>
"@
        
        foreach ($metric in $thresholds.Keys) {
            $value = $sample.Metrics.$metric
            $health = $sample.Health[$metric]
            @"
                    <td><span class="$health">$value</span></td>
"@
        }
        
        @"
                </tr>
"@
    }
    
    @"
            </table>
        </div>
    </div>
"@
}

$htmlFooter = @"
</body>
</html>
"@

# Combine and save HTML
$htmlOutput = $htmlHeader + $htmlContent + $htmlFooter
$outputPath = "ServerHealthDashboard.html"
$htmlOutput | Out-File -FilePath $outputPath -Encoding UTF8

Write-Host "Dashboard generated: $outputPath"
Previous Post Next Post

نموذج الاتصال