PowerShell: Speed Up Get-ChildItem to Scan Files in a Large File System

I wrote a PowerShell script for scanning PST files in a large file system. It took about 19–21 hours to scan PST files in the 28 TB file system.

The Slow Way

$stamp = Get-Date

$location = "\\deptshare\shares"
$csvfile  = "C:\temp\pstfiles\" + $(Get-Date -Format yyyyMMddHmm) + "deptshare-pst.csv"
$pstarray = @()

ForEach ($share in Get-ChildItem $location) {
    Write-Host "Scanning $share..."
    Get-ChildItem $share.FullName -recurse -file -force -ErrorAction SilentlyContinue |
        Where-Object { $_.extension -eq ".pst" } |
        ForEach-Object {
            $pstobj = New-Object PSObject
            $pstobj | Add-Member NoteProperty Directory $_.DirectoryName
            $pstobj | Add-Member NoteProperty Name $_.Name
            $pstobj | Add-Member NoteProperty Owner ((Get-ACL $_.FullName).Owner)
            $pstobj | Add-Member NoteProperty CreationTime $_.CreationTime
            $pstobj | Add-Member NoteProperty LastWriteTime $_.LastWriteTime
            $pstobj | Add-Member NoteProperty Length $_.Length
            $pstarray += $pstobj
        }
}

The key slowdown was the Get-ChildItem cmdlet combined with Where-Object.

Measure-Command Experiment (60 GB scan)

Old way — took 307 seconds:

Get-ChildItem $share.FullName -recurse -file -force -ErrorAction SilentlyContinue |
    Where-Object { $_.extension -eq ".pst" }

New way — took only 19 seconds:

Get-ChildItem $share.FullName -recurse -file -force -Include *.pst -ErrorAction SilentlyContinue

Using -Include *.pst instead of Where-Object { $_.extension -eq ".pst" } provides a massive speed improvement.

After applying this change in production, the scan time reduced to 14 hours from 21 hours. Massive improvement!