Powershell : Automating Linux-to-Windows File Transfers

The goal here is to from Powershell is to remotely run commands and then collect report and data from Linux servers and make them available on Windows infrastructures. This blog post explores a practical solution using PowerShell and PuTTY's PSCP tool to automate the process of connecting to a Linux server, executing commands if needed, and transferring HTML reports to a Windows SMB share.

The Challenge

Our specific use case involves:

1. Securely connecting to a remote Linux server
2. Remotely executing Python scripts to generate/update the data
3. Triggering the HTML dashboard report generation
4. Retrieving the HTML dashboard report
5. Transferring it to a specific Windows SMB network share
6. Ensuring the file properly overwrites any existing version

The Approach

I have used PowerShell with PuTTY's PSCP (PuTTY Secure Copy Protocol) utility to handle the secure file transfer. The script includes credential management, error handling, and verification steps to ensure reliability.

Script : SSLTransfer.ps1

$PuttyPath = "C:\Program Files\PuTTY\pscp.exe"
$PlinkPath = "C:\Program Files\PuTTY\plink.exe"
$CredentialFile = "sslcredentials.xml"

# Import credentials from XML
$Credential = Import-Clixml -Path $CredentialFile
$Username = $Credential.UserName
$Password = $Credential.GetNetworkCredential().Password

# Define the remote server, file path, and destination
$RemoteHost = "10.80.44.10"
$RemotePath = "/home/mooney/ssldata/"
$RemoteFile = "ssl_dashboard.html"
$LocalTempPath = "$env:TEMP\ssl_temp\"
$LocalTempFile = "$LocalTempPath$RemoteFile"
$DestinationPath = "\\bearsmb01\iisroot$\ssl\External\index.html"

# Create temp directory if it doesn't exist
if (-not (Test-Path $LocalTempPath)) {
    New-Item -Path $LocalTempPath -ItemType Directory -Force | Out-Null
}

# Clean up any existing temporary files
if (Test-Path $LocalTempFile) {
    Remove-Item -Path $LocalTempFile -Force
}

# Step 1: Execute the first Python script to collect data
Write-Host "Executing data collection script on remote server..."
$DataCollectionCommand = "cd /home/mooney/ssldata && python3 collect_ssl_data.py"
& $PlinkPath -ssh -l $Username -pw $Password $RemoteHost $DataCollectionCommand

# Check the exit code to determine if the command was successful
if ($LASTEXITCODE -eq 0) {
    Write-Host "Data collection completed successfully."
} else {
    Write-Host "ERROR: Data collection failed with exit code $LASTEXITCODE"
    exit $LASTEXITCODE
}

# Step 2: Execute the second Python script to process data
Write-Host "Processing collected data on remote server..."
$DataProcessingCommand = "cd /home/mooney/ssldata && python3 process_ssl_data.py"
& $PlinkPath -ssh -l $Username -pw $Password $RemoteHost $DataProcessingCommand

# Check the exit code to determine if the command was successful
if ($LASTEXITCODE -eq 0) {
    Write-Host "Data processing completed successfully."
} else {
    Write-Host "ERROR: Data processing failed with exit code $LASTEXITCODE"
    exit $LASTEXITCODE
}

# Step 3: Generate the HTML dashboard
Write-Host "Generating HTML dashboard report..."
$GenerateReportCommand = "cd /home/mooney/ssldata && python3 generate_ssl_dashboard.py"
& $PlinkPath -ssh -l $Username -pw $Password $RemoteHost $GenerateReportCommand

# Check the exit code to determine if the command was successful
if ($LASTEXITCODE -eq 0) {
    Write-Host "HTML dashboard generated successfully."
} else {
    Write-Host "ERROR: Dashboard generation failed with exit code $LASTEXITCODE"
    exit $LASTEXITCODE
}

# Download the file from the remote server
Write-Host "Transferring $RemoteFile from $RemoteHost to local temp location..."
& $PuttyPath -q -scp -l $Username -pw $Password $RemoteHost`:$RemotePath$RemoteFile $LocalTempFile

# Ensure the transfer was successful before copying to destination
if (Test-Path $LocalTempFile) {
    Write-Host "File downloaded successfully to: $LocalTempFile"
    
    # Ensure destination directory exists
    $DestinationFolder = Split-Path -Path $DestinationPath -Parent
    if (-not (Test-Path $DestinationFolder)) {
        New-Item -Path $DestinationFolder -ItemType Directory -Force | Out-Null
        Write-Host "Created destination directory: $DestinationFolder"
    }
    
    # Copy the file to the destination, overwriting if it exists
    Write-Host "Copying file to destination: $DestinationPath"
    Copy-Item -Path $LocalTempFile -Destination $DestinationPath -Force
    
    if (Test-Path $DestinationPath) {
        Write-Host "File successfully copied to: $DestinationPath"
    } else {
        Write-Host "ERROR: Failed to copy file to destination path"
    }
} else {
    Write-Host "ERROR: Transfer failed for $RemoteFile from $RemoteHost"
}

# Clean up temp files
if (Test-Path $LocalTempPath) {
    Remove-Item -Path $LocalTempPath -Recurse -Force
    Write-Host "Temporary files cleaned up"
}

Write-Host "Transfer operation completed."

Code : Detailed Breakdown

Let's break down each section of the script to understand exactly how it works:

1. Setup and Configuration

$PuttyPath = "C:\Program Files\PuTTY\pscp.exe"
$CredentialFile = "sslcredentials.xml"

• $PuttyPath: Specifies the path to the PSCP executable, which is part of the PuTTY suite of tools
• $CredentialFile: Points to an XML file containing encrypted credentials

2. Credential Management

# Import credentials from XML
$Credential = Import-Clixml -Path $CredentialFile
$Username = $Credential.UserName
$Password = $Credential.GetNetworkCredential().Password

• Import-Clixml: Securely imports the encrypted credential object from the XML file
• The credential object contains both the username and an encrypted password
• GetNetworkCredential().Password: Safely decrypts the password for use in the script

Security Note: The credential file must be created beforehand using Get-Credential | Export-Clixml -Path "sslcredentials.xml" and can only be decrypted by the same user account that created it on the same computer.

3. Configuration Variables

$PlinkPath = "C:\Program Files\PuTTY\plink.exe"
# Define the remote server, file path, and destination
$RemoteHost = "10.80.44.10"
$RemotePath = "/home/mooney/ssldata/"
$RemoteFile = "ssl_dashboard.html"
$LocalTempPath = "$env:TEMP\ssl_temp\"
$LocalTempFile = "$LocalTempPath$RemoteFile"
$DestinationPath = "\\bearsmb01\iisroot$\ssl\External\index.html"

• $PlinkPath: Path to the Plink executable, which is used for remote command execution
• $RemoteHost: The IP address of the Linux server
• $RemotePath: The directory path on the remote server where the file is located
• $RemoteFile: The name of the file to transfer
• $LocalTempPath: A temporary directory on the local Windows machine (using the system temp directory)
• $LocalTempFile: The full path to the temporary file location
• $DestinationPath: The UNC path to the Windows SMB share where the file will be copied

4. Temporary Directory Management

# Create temp directory if it doesn't exist
if (-not (Test-Path $LocalTempPath)) {
    New-Item -Path $LocalTempPath -ItemType Directory -Force | Out-Null
}

# Clean up any existing temporary files
if (Test-Path $LocalTempFile) {
    Remove-Item -Path $LocalTempFile -Force
}

• Test-Path: Checks if the temporary directory already exists
• New-Item: Creates the directory if it doesn't exist
• Out-Null: Suppresses output from the directory creation
• The script also removes any existing temporary file to ensure a clean transfer

5. Remote Python Script Execution

# Step 1: Execute the first Python script to collect data
Write-Host "Executing data collection script on remote server..."
$DataCollectionCommand = "cd /home/mooney/ssldata && python3 collect_ssl_data.py"
& $PlinkPath -ssh -l $Username -pw $Password $RemoteHost $DataCollectionCommand

# Check the exit code to determine if the command was successful
if ($LASTEXITCODE -eq 0) {
    Write-Host "Data collection completed successfully."
} else {
    Write-Host "ERROR: Data collection failed with exit code $LASTEXITCODE"
    exit $LASTEXITCODE
}

• &: The call operator, which runs the Plink executable
• -ssh: Specifies SSH protocol for the remote connection
• -l $Username: Specifies the login username
• -pw $Password: Provides the password
• $RemoteHost: The address of the Linux server
• $DataCollectionCommand: The command to run on the remote server
• $LASTEXITCODE: PowerShell variable that contains the exit code of the last command
• This section executes the Python script that collects the raw SSL data

# Step 2: Execute the second Python script to process data
Write-Host "Processing collected data on remote server..."
$DataProcessingCommand = "cd /home/mooney/ssldata && python3 process_ssl_data.py"
& $PlinkPath -ssh -l $Username -pw $Password $RemoteHost $DataProcessingCommand

• This section runs the script that processes the collected data, performing analysis and transformations

# Step 3: Generate the HTML dashboard
Write-Host "Generating HTML dashboard report..."
$GenerateReportCommand = "cd /home/mooney/ssldata && python3 generate_ssl_dashboard.py"
& $PlinkPath -ssh -l $Username -pw $Password $RemoteHost $GenerateReportCommand

• This final command execution generates the actual HTML dashboard from the processed data

6. File Transfer from Linux Server

# Download the file from the remote server
Write-Host "Transferring $RemoteFile from $RemoteHost to local temp location..."
& $PuttyPath -q -scp -l $Username -pw $Password $RemoteHost`:$RemotePath$RemoteFile $LocalTempFile

• &: The call operator, which runs the PSCP executable
• -q: Quiet mode, suppresses non-essential output
• -scp: Uses SCP protocol for file transfer
• -l $Username: Specifies the login username
• -pw $Password: Provides the password (note: this is generally discouraged for security but is acceptable in an automated script using encrypted credentials)
• $RemoteHost:$RemotePath$RemoteFile`: The full remote path in the format required by PSCP
• $LocalTempFile: The local destination for the file

7. Verification and Destination Copy

# Ensure the transfer was successful before copying to destination
if (Test-Path $LocalTempFile) {
    Write-Host "File downloaded successfully to: $LocalTempFile"
    
    # Ensure destination directory exists
    $DestinationFolder = Split-Path -Path $DestinationPath -Parent
    if (-not (Test-Path $DestinationFolder)) {
        New-Item -Path $DestinationFolder -ItemType Directory -Force | Out-Null
        Write-Host "Created destination directory: $DestinationFolder"
    }
    
    # Copy the file to the destination, overwriting if it exists
    Write-Host "Copying file to destination: $DestinationPath"
    Copy-Item -Path $LocalTempFile -Destination $DestinationPath -Force
    
    if (Test-Path $DestinationPath) {
        Write-Host "File successfully copied to: $DestinationPath"
    } else {
        Write-Host "ERROR: Failed to copy file to destination path"
    }
} else {
    Write-Host "ERROR: Transfer failed for $RemoteFile from $RemoteHost"
}

• The script first verifies that the file was successfully downloaded
• Split-Path: Extracts just the directory portion of the destination path
• The script checks if the destination directory exists and creates it if necessary
• Copy-Item with -Force: Copies the file and overwrites any existing file
• A final verification confirms the file was successfully copied to the destination

8. Cleanup

# Clean up temp files
if (Test-Path $LocalTempPath) {
    Remove-Item -Path $LocalTempPath -Recurse -Force
    Write-Host "Temporary files cleaned up"
}

Write-Host "Transfer operation completed."

• Remove-Item with -Recurse: Removes the entire temporary directory and its contents
• The script confirms completion of the operation

Order of Processing: Remote Python Script Execution

Before transferring the HTML dashboard, I needed to execute Python scripts on the remote Linux server to collect and process the data and here I have used PuTTY's plink.exe tool for this purpose.

# Path to Plink executable
$PlinkPath = "C:\Program Files\PuTTY\plink.exe"

# Step 1: Execute the first Python script to collect data
Write-Host "Executing data collection script on remote server..."
$DataCollectionCommand = "cd /home/mooney/ssldata && python3 collect_ssl_data.py"
& $PlinkPath -ssh -l $Username -pw $Password $RemoteHost $DataCollectionCommand

# Check the exit code to determine if the command was successful
if ($LASTEXITCODE -eq 0) {
    Write-Host "Data collection completed successfully."
} else {
    Write-Host "ERROR: Data collection failed with exit code $LASTEXITCODE"
    exit $LASTEXITCODE
}

# Step 2: Execute the second Python script to process data
Write-Host "Processing collected data on remote server..."
$DataProcessingCommand = "cd /home/mooney/ssldata && python3 process_ssl_data.py"
& $PlinkPath -ssh -l $Username -pw $Password $RemoteHost $DataProcessingCommand

# Check the exit code to determine if the command was successful
if ($LASTEXITCODE -eq 0) {
    Write-Host "Data processing completed successfully."
} else {
    Write-Host "ERROR: Data processing failed with exit code $LASTEXITCODE"
    exit $LASTEXITCODE
}

# Step 3: Generate the HTML dashboard
Write-Host "Generating HTML dashboard report..."
$GenerateReportCommand = "cd /home/mooney/ssldata && python3 generate_ssl_dashboard.py"
& $PlinkPath -ssh -l $Username -pw $Password $RemoteHost $GenerateReportCommand

# Check the exit code to determine if the command was successful
if ($LASTEXITCODE -eq 0) {
    Write-Host "HTML dashboard generated successfully."
} else {
    Write-Host "ERROR: Dashboard generation failed with exit code $LASTEXITCODE"
    exit $LASTEXITCODE
}

These commands would run before the file transfer section and execute the specified Python scripts on the remote server. Each step:

1. Collects raw SSL certificate data from various sources
2. Processes and analyzes the collected data
3. Generates the final HTML dashboard report

The script checks the exit code after each command to ensure it was successful before proceeding to the next step. This ensures that the script does not attempt to transfer an incomplete or corrupted HTML report.

Scheduling The Script

For true automation, you can schedule this script using Task Scheduler this means you can get the script to run between the frequency, depending on your requirements of hourly or daily.

Furthermore, with the script being run from windows, you do not have to handle Windows authentication negotiation from a foreign host that is not in the domain .

Security Considerations

Obviously, it’s a good idea to restrict access and script execution permissions to authorize users only, furthermore, the account used to run the python scripts has just enough access to get the job done, ensuring in short, compliance with the least privileged access methodology.

1. The credential file is encrypted using Windows Data Protection API (DPAPI) and can only be decrypted by the same user on the same machine
2. Restrict permissions on the script and credential file to only authorized users
3. Use a service account with minimal required permissions for the file.