Tuesday, June 23, 2015

Powershell Script - Invoke-Multithread - Make any script multithreaded

Here's a very useful wrapper which helps you turn your single threaded sequential functions into multithreaded beasts which complete in a fraction of the time when addressing many servers (or whatever network devices).

The function works by taking your code as a [scriptblock] object and a list of computers (servers, IPs, whatever) and breaking up the list based on a throttle limit (default is 25) and starting powershell jobs.  The script waits for each job to finish and captures the return from the job and parses it into an object array and associates each computer's output with the name.

This function is great for generating reports, pushing configurations, making changes, etc, where native multithreaded cmdlets won't work and sequential code must be used.  Most admins new to powershell don't know how to leverage built in commands that automatically thread and this wrapper function does a nice job at speeding up slow scripts.


#Requires -Version 3.0
Function Invoke-Multithread{
<#
.SYNOPSIS
Wrapper function that lets you divide and conquer a server list to multithread it's execution.
.DESCRIPTION
Similar behavior to Invoke-Command -Asjob which lets you remotely start scriptblocks on target servers,
this function starts local jobs that are to be targeted at remote servers.
Great for multithreaded push deployments or report generation
.PARAMETER ComputerName
Specifies one or more server names.
.PARAMETER jobheader
Optional prefix name for jobs, so you can use more than one Invoke-Multithread on the same machine
.PARAMETER ThrottleLimit
Limits the number of jobs started. If you have 1000 servers, the default limit of 25 will mean you'll get 25 jobs, each with 40 servers per job.
Don't see the limit too large as it will eat up more CPU and RAM which actually might cause things to run slower than executing sequentially.
.EXAMPLE
Invoke-Multithread -ComputerName (Get-Content .\myserverlist.txt) -ScriptBlock{<your script here, with -computername as a parameter>}
Starts up to 25 threads of your code for local execution and returns results back in an object array with computer names
.LINK
http://www.bryanvine.com/2015/06/powershell-script-invoke-multithread.html
.LINK
Start-Job
.LINK
Get-Job
.LINK
Receive-Job
.LINK
Invoke-Command
.NOTES
Author: Bryan Vine
Last updated: 6/23/2015
#>
[CmdletBinding()]
Param(
[Parameter(Mandatory=$true,valuefrompipeline=$false,Position=0)]
[alias("CN","MachineName","ServerName","Server")][ValidateNotNullOrEmpty()][string[]]
$ComputerName,
[Parameter(Mandatory=$true)][scriptblock]$ScriptBlock,
[string]$jobheader='Multi',
[int]$ThrottleLimit = 25
)
BEGIN{
$jobs = 1
$i = 0
#Clear out jobs from previous runs
Get-Job "$jobheader*"| Remove-Job -Force
}
PROCESS{
#Calculates number of computers per thread
$BatchSize = [math]::Round($ComputerName.count / $ThrottleLimit,0) + 1
#loop while there's still work to be done
while($jobs -and $i -lt $ComputerName.count){
#return jobs that are finished and parses output per computer name as nice objects
Get-Job "$jobheader*" | ?{$_.State -like "completed"}|%{
$servers = ($_ | select -ExpandProperty Command).replace($ScriptBlock.ToString(),"").replace("-ComputerName ('","").replace("').split(',')","").split(",")
$jobreturn = $_ | Receive-Job
0..($servers.count -1) | %{
$obj = New-Object PSobject
$obj | Add-Member NoteProperty ComputerName $servers[$_]
$obj | Add-Member NoteProperty Output $jobreturn[$_]
Write-Output $obj
}
}
if($jobs -lt $ThrottleLimit){
#spawn more jobs
#break up server list into batches to kick off multiple jobs
$batch = @()
1..$BatchSize | %{
$batch += $ComputerName[$i++] |?{$_ -notlike ''}
}
if($batch){
$batchstring = $batch -join ","
#start the job by appending the -computername property with the servernames to the scriptblock
Start-Job -Name "$jobheader-$i" -ScriptBlock ([scriptblock]::Create("$($ScriptBlock.ToString()) -ComputerName ('$batchstring').split(',')")) | Out-Null
}
}
else{
Start-Sleep -Seconds 1
}
#calculate the number of jobs currently running
$jobs = (Get-Job "$jobheader*" |?{$_.State -like "Running"})| Measure | select -ExpandProperty count
}
}
END{
#return all jobs after completed
Get-Job "$jobheader*" | Wait-Job | %{
$servers = ($_ | select -ExpandProperty Command).replace($ScriptBlock.ToString(),"").replace(" -ComputerName ('","").replace("').split(',')","").split(",")
$jobreturn = $_ | Receive-Job
0..($servers.count -1) | %{
$obj = New-Object PSobject
$obj | Add-Member NoteProperty ComputerName $servers[$_]
$obj | Add-Member NoteProperty Output $jobreturn[$_]
Write-Output $obj
}
}
#cleanup
Get-Job "$jobheader*"| Remove-Job
}
}
Here's an example where I'm just checking to see if the server is a windows server, but this could be any function which takes a string array "ComputerName" as a parameter.

#http://www.bryanvine.com/2015/06/powershell-script-invoke-multithread.html
#ScriptBlock: function declaration with function name on the last line without the -ComputerName parameter
$SB = {
Function Check-WindowsServer{
param(
[string[]]$ComputerName
)
foreach ($server in $ComputerName){
if(Test-Path "\\$server\c$\windows"){
Write-Output "Is a Windows Server"
}
else{
Write-Output "Is NOT a Windows Server"
}
}
}
Check-WindowsServer}
Invoke-Multithread -ComputerName (Get-Content .\myserverlist.txt) -ScriptBlock $SB
view raw gistfile1.ps1 hosted with ❤ by GitHub
And the output:

ComputerName  Output
------------  ------
SERVER1       Is a Windows Server
SERVER2       Is a Windows Server
SERVER3       Is a Windows Server
SERVER4       Is a Windows Server
SERVER5       Is a Windows Server
SERVER6       Is a Windows Server
SERVER7       Is a Windows Server
SERVER8       Is a Windows Server
SERVER9       Is a Windows Server
SERVER10      Is a Windows Server
SERVER11      Is a Windows Server
SERVER12      Is a Windows Server
SERVER13      Is a Windows Server
SERVER14      Is a Windows Server
SERVER15      Is a Windows Server
SERVER16      Is a Windows Server
SERVER17      Is a Windows Server
SERVER18      Is a Windows Server
SERVER19      Is a Windows Server
SERVER20      Is a Windows Server
SERVER21      Is a Windows Server
SERVER22      Is a Windows Server
SERVER23      Is a Windows Server
SERVER24      Is a Windows Server
SERVER25      Is a Windows Server
SERVER26      Is a Windows Server
SERVER27      Is a Windows Server
SERVER28      Is a Windows Server
SERVER29      Is a Windows Server
SERVER30      Is a Windows Server
SERVER31      Is a Windows Server
SERVER32      Is a Windows Server
SERVER33      Is a Windows Server
SERVER34      Is a Windows Server
SERVER35      Is a Windows Server
SERVER36      Is a Windows Server
SERVER37      Is a Windows Server
SERVER38      Is a Windows Server
SERVER39      Is a Windows Server
SERVER40      Is a Windows Server
SERVER41      Is a Windows Server
SERVER42      Is a Windows Server
SERVER43      Is a Windows Server
SERVER44      Is a Windows Server
SERVER45      Is a Windows Server
SERVER46      Is a Windows Server
SERVER47      Is a Windows Server
SERVER48      Is a Windows Server
SERVER49      Is a Windows Server
SERVER50      Is a Windows Server
NAS1          Is NOT a Windows Server

No comments: